Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-65278

Random but usual behat failures with submission_types.feature

    XMLWordPrintable

    Details

    • Affected Branches:
      MOODLE_36_STABLE, MOODLE_37_STABLE

      Description

      Since some time ago we are detecting that the scenarios @ mod/workshop/tests/behat/submission_types.feature do fail randomly, but often than desired. That leads to lots of "yellows" and some "reds" in automated testing results. We should aim for more "greens" here.

      We have noticed that the problems happen often when CI workers are really busy and also under Oracle (which 2nd name is slowness). That may indicate that there is some point in the escenarios where behat is not waiting properly (by searching for something) or the standard js_pending()/js_complete() stuff is missing.

      Here there are the usual errors:

      Link: https://ci.moodle.org/job/S36.03.05%20-%20Behat%20-%20Chrome%20+%20Oracle/58/

      (you can see that 3 different failed and one of them failed twice, so the run ended red)

      Output:

      001 Scenario: Online text optional, file attachment required # /var/www/html/mod/workshop/tests/behat/submission_types.feature:110
            And I press "Start preparing your submission"          # /var/www/html/mod/workshop/tests/behat/submission_types.feature:118
              Button matching locator "'Start preparing your submission'" not found. (Behat\Mink\Exception\ElementNotFoundException)
       
      002 Scenario: File attachment only                   # /var/www/html/mod/workshop/tests/behat/submission_types.feature:150
            When I press "Start preparing your submission" # /var/www/html/mod/workshop/tests/behat/submission_types.feature:158
              Button matching locator "'Start preparing your submission'" not found. (Behat\Mink\Exception\ElementNotFoundException)
       
      003 Scenario: Neither submission type explicitly required # /var/www/html/mod/workshop/tests/behat/submission_types.feature:172
            And I press "Start preparing your submission"       # /var/www/html/mod/workshop/tests/behat/submission_types.feature:180
              Button matching locator "'Start preparing your submission'" not found. (Behat\Mink\Exception\ElementNotFoundException)
       
      420 scenarios (417 passed, 3 failed)
      9994 steps (9958 passed, 3 failed, 33 skipped)
      237m4.01s (107.13Mb)
      

      Ciao

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              stronk7 Eloy Lafuente (stronk7)
              Participants:
              Component watchers:
              Andrew Nicols, Mathew May, Michael Hawkins, Shamim Rezaie, Simey Lameze, Amaia Anabitarte, Bas Brands, Carlos Escobedo, Sara Arjona (@sarjona), Víctor Déniz Falcón
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated: