Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-69455

BeforeScenario hook failure leads to skipped test and rerun (take #2)

    XMLWordPrintable

Details

    • Bug
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • 3.5.13, 3.7.7, 3.8.4, 3.9.1, 3.10
    • None
    • None
    • MOODLE_310_STABLE, MOODLE_35_STABLE, MOODLE_37_STABLE, MOODLE_38_STABLE, MOODLE_39_STABLE

    Description

      This is a followup of MDL-69278 where it was detected that, sometimes, problem in BeforeScenario stuff leads to skipped tests (not failures), but the run ends with non-zero exit code.

      That causes reruns to happen and, as far as there isn't any rerun list (with failures) the whole job is rerun.

      In MDL-69278 it was fixed for JS scenarios, but it seems that some "goutte" BeforeScenario stuff also can lead to the very same problem, increasing execution a lot.

      Just got this case:

      https://ci.moodle.org/job/S37.07.01%20-%20Behat%20-%20Firefox%20+%20Postgres%20+%20Classic/177/

      • Normal (javascript) postgres run.
      • 2 of the 3 parallel runs end with:

            BeforeScenario @~javascript # behat_hooks::before_goutte_scenarios()
              Could not open connection: Timed out waiting 45 seconds for Firefox to start.
              Build info: version: '3.141.59', revision: 'e82be7d358', time: '2018-11-14T08:25:53'
              System info: host: 'ec15f4ddf0d7', ip: '172.27.0.6', os.name: 'Linux', os.arch: 'amd64', os.version: '4.15.0-107-generic', java.version: '1.8.0_222'
              Driver info: driver.version: unknown (Behat\Mink\Exception\DriverException)
        

      • That leads to 1 skipped, 0 failed. And exit error.
      • Rerun reruns everything (waste of resources, 10h total run time).

      So, pretty much the original issue, but happening on the before_goutte_scenarios(), for runs that aren't goutte.

      Surely they deserve a similar/parallel fix (or understand why that is being called).

      Ciao

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              stronk7 Eloy Lafuente (stronk7)
              Andrew Lyons, Huong Nguyen, Jun Pataleta, Michael Hawkins, Shamim Rezaie, Simey Lameze, Stevani Andolo
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated: