Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-69455

BeforeScenario hook failure leads to skipped test and rerun (take #2)

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 3.5.13, 3.7.7, 3.8.4, 3.9.1, 3.10
    • Fix Version/s: None
    • Labels:
      None
    • Affected Branches:
      MOODLE_310_STABLE, MOODLE_35_STABLE, MOODLE_37_STABLE, MOODLE_38_STABLE, MOODLE_39_STABLE

      Description

      This is a followup of MDL-69278 where it was detected that, sometimes, problem in BeforeScenario stuff leads to skipped tests (not failures), but the run ends with non-zero exit code.

      That causes reruns to happen and, as far as there isn't any rerun list (with failures) the whole job is rerun.

      In MDL-69278 it was fixed for JS scenarios, but it seems that some "goutte" BeforeScenario stuff also can lead to the very same problem, increasing execution a lot.

      Just got this case:

      https://ci.moodle.org/job/S37.07.01%20-%20Behat%20-%20Firefox%20+%20Postgres%20+%20Classic/177/

      • Normal (javascript) postgres run.
      • 2 of the 3 parallel runs end with:

            BeforeScenario @~javascript # behat_hooks::before_goutte_scenarios()
              Could not open connection: Timed out waiting 45 seconds for Firefox to start.
              Build info: version: '3.141.59', revision: 'e82be7d358', time: '2018-11-14T08:25:53'
              System info: host: 'ec15f4ddf0d7', ip: '172.27.0.6', os.name: 'Linux', os.arch: 'amd64', os.version: '4.15.0-107-generic', java.version: '1.8.0_222'
              Driver info: driver.version: unknown (Behat\Mink\Exception\DriverException)
        

      • That leads to 1 skipped, 0 failed. And exit error.
      • Rerun reruns everything (waste of resources, 10h total run time).

      So, pretty much the original issue, but happening on the before_goutte_scenarios(), for runs that aren't goutte.

      Surely they deserve a similar/parallel fix (or understand why that is being called).

      Ciao

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              stronk7 Eloy Lafuente (stronk7)
              Participants:
              Component watchers:
              Andrew Lyons, Dongsheng Cai, Huong Nguyen, Jun Pataleta, Michael Hawkins, Shamim Rezaie, Simey Lameze
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Dates

                Created:
                Updated: