This is a followup of MDL-69278 where it was detected that, sometimes, problem in BeforeScenario stuff leads to skipped tests (not failures), but the run ends with non-zero exit code.

That causes reruns to happen and, as far as there isn't any rerun list (with failures) the whole job is rerun.

In MDL-69278 it was fixed for JS scenarios, but it seems that some "goutte" BeforeScenario stuff also can lead to the very same problem, increasing execution a lot.

Just got this case:

https://ci.moodle.org/job/S37.07.01%20-%20Behat%20-%20Firefox%20+%20Postgres%20+%20Classic/177/

• Normal (javascript) postgres run.
• 2 of the 3 parallel runs end with:

  BeforeScenario @~javascript # behat_hooks::before_goutte_scenarios()

  Could not open connection: Timed out waiting 45 seconds for Firefox to start.

  Build info: version: '3.141.59', revision: 'e82be7d358', time: '2018-11-14T08:25:53'

  System info: host: 'ec15f4ddf0d7', ip: '172.27.0.6', os.name: 'Linux', os.arch: 'amd64', os.version: '4.15.0-107-generic', java.version: '1.8.0_222'

  Driver info: driver.version: unknown (Behat\Mink\Exception\DriverException)

• That leads to 1 skipped, 0 failed. And exit error.
• Rerun reruns everything (waste of resources, 10h total run time).

So, pretty much the original issue, but happening on the before_goutte_scenarios(), for runs that aren't goutte.

Surely they deserve a similar/parallel fix (or understand why that is being called).

Ciao