This may be related with MDL-67175 where it was detected that incoming Chome 80 would be breaking some SCORM tests.

That issue was integrated some days ago but, no matter of that, our testing infrastructure has started to get random (they don't happen always, although sometimes they happen twice) failures related with SCORM. All branches. Chrome only.

And the begin of the failures matches closely the release of selenium 3.141.59-zirconium, few days ago, the very first coming with Chrome 80.

Basically we are getting these 2 scenarios failing (again, not always, but really often):

• Test force completed set to when previous complete/passed/failed. : 25 times (happens twice 8 times)
• Test review mode with all scos completed. : 5 times (happens twice 2 times)
• Test completion with all scos and correct sco load on re-entry : 4 times (happens twice 2 times)

Here there are some links:

• https://ci.moodle.org/job/W.03.03%20-%20Behat%20-%20Chrome%20+%20MariaDB%20(lowest)/545/
• https://ci.moodle.org/job/W36.03.03%20-%20Behat%20-%20Chrome%20+%20MariaDB%20(lowest)/259/
• https://ci.moodle.org/job/W.03.05%20-%20Behat%20-%20Chrome%20+%20Oracle/514/

Side note, to avoid all the noise that these failures are causing, we are going to pin our selenium-chrome to the previous selenium 3.141.59-zinc version.

PR to downgrade selenium to Chrome 79: https://github.com/moodlehq/moodle-ci-runner/pull/22
Issue to unpin Chrome 79 (once this is fixed): https://github.com/moodlehq/moodle-ci-runner/issues/21