Affects Version/s: 3.5.13, 3.7.7, 3.8.4, 3.9.1, 3.10
Fix Version/s: None
Component/s: Automated functional tests (behat)
Affected Branches:MOODLE_310_STABLE, MOODLE_35_STABLE, MOODLE_37_STABLE, MOODLE_38_STABLE, MOODLE_39_STABLE
This is a followup of
MDL-69278 where it was detected that, sometimes, problem in BeforeScenario stuff leads to skipped tests (not failures), but the run ends with non-zero exit code.
That causes reruns to happen and, as far as there isn't any rerun list (with failures) the whole job is rerun.
MDL-69278 it was fixed for JS scenarios, but it seems that some "goutte" BeforeScenario stuff also can lead to the very same problem, increasing execution a lot.
Just got this case:
- 2 of the 3 parallel runs end with:
- That leads to 1 skipped, 0 failed. And exit error.
- Rerun reruns everything (waste of resources, 10h total run time).
So, pretty much the original issue, but happening on the before_goutte_scenarios(), for runs that aren't goutte.
Surely they deserve a similar/parallel fix (or understand why that is being called).