Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-66675

Add pause on fail option to behat

    XMLWordPrintable

    Details

    • Testing Instructions:
      Hide

      Test

      1. Drop the attached 66675.feature into lib/tests/behat
      2. Initialise behat
      3. Run just that feature file
        1. Confirm that the first test was paused after a few steps
      4. Press [Enter/return]
        1. Confirm that the test continue running
        2. Confirm that the dots for the test continued on the line below the Enter message
        3. Confirm that the dots started straight after the previous set finished
      5. The test will continue running until...
        1. Confirm that the test failed
        2. Confirm that the reason for the failure was shown
        3. Confirm that you the test was not paused
      6. Edit your config.php and set:

        $CFG->behat_pause_on_fail = true;
        

      7. Run just that feature file again
        1. Confirm that the test paused in the same place as before
      8. Continue the test
      9. The test will continue running until...
        1. Confirm that the test failed
        2. Confirm that the reason for the failure was shown
        3. Confirm that the test paused at the failure point
      10. Press [Enter/return]
        1. Confirm that the rest of the steps were skipped running
        2. Confirm that the dots for the test continued on the line below the Enter message
        3. Confirm that the dots started straight after the previous set finished

      Example output

      Started at 19-09-2019, 07:37
      ........
      Paused. Press Enter/Return to continue.
              ...............
      Scenario failed. Paused for inspection. Press Enter/Return to continue.
      Exception follows:
      Link matching locator "Participants" not found.
                             F---
       
      --- Failed steps:
       
      001 Scenario: A test fails automatically on a failure # /Users/nicols/Sites/moodles/sm/moodle/lib/tests/behat/66675.feature:38
            When I follow "Participants"                    # /Users/nicols/Sites/moodles/sm/moodle/lib/tests/behat/66675.feature:42
              Link matching locator "Participants" not found. (Behat\Mink\Exception\ElementNotFoundException)
       
      2 scenarios (1 passed, 1 failed)
      27 steps (23 passed, 1 failed, 3 skipped)
      3m0.05s (52.70Mb)
      

      Show
      Test Drop the attached 66675.feature into lib/tests/behat Initialise behat Run just that feature file Confirm that the first test was paused after a few steps Press [Enter/return] Confirm that the test continue running Confirm that the dots for the test continued on the line below the Enter message Confirm that the dots started straight after the previous set finished The test will continue running until... Confirm that the test failed Confirm that the reason for the failure was shown Confirm that you the test was not paused Edit your config.php and set: $CFG->behat_pause_on_fail = true; Run just that feature file again Confirm that the test paused in the same place as before Continue the test The test will continue running until... Confirm that the test failed Confirm that the reason for the failure was shown Confirm that the test paused at the failure point Press [Enter/return] Confirm that the rest of the steps were skipped running Confirm that the dots for the test continued on the line below the Enter message Confirm that the dots started straight after the previous set finished Example output Started at 19-09-2019, 07:37 ........ Paused. Press Enter/Return to continue. ............... Scenario failed. Paused for inspection. Press Enter/Return to continue. Exception follows: Link matching locator "Participants" not found. F---   --- Failed steps:   001 Scenario: A test fails automatically on a failure # /Users/nicols/Sites/moodles/sm/moodle/lib/tests/behat/66675.feature:38 When I follow "Participants" # /Users/nicols/Sites/moodles/sm/moodle/lib/tests/behat/66675.feature:42 Link matching locator "Participants" not found. (Behat\Mink\Exception\ElementNotFoundException)   2 scenarios (1 passed, 1 failed) 27 steps (23 passed, 1 failed, 3 skipped) 3m0.05s (52.70Mb)
    • Affected Branches:
      MOODLE_35_STABLE, MOODLE_36_STABLE, MOODLE_37_STABLE, MOODLE_38_STABLE
    • Fixed Branches:
      MOODLE_35_STABLE, MOODLE_36_STABLE, MOODLE_37_STABLE
    • Pull 3.5 Branch:
    • Pull 3.7 Branch:
    • Pull Master Branch:
      MDL-66675-master

      Description

      When tracking down behat fails it's helpful to be able to pause when the test fails.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              dobedobedoh Andrew Nicols
              Reporter:
              dobedobedoh Andrew Nicols
              Peer reviewer:
              Sam Marshall
              Integrator:
              Eloy Lafuente (stronk7)
              Tester:
              Janelle Barcega
              Participants:
              Component watchers:
              Andrew Nicols, Jun Pataleta, Michael Hawkins, Shamim Rezaie, Simey Lameze
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:
                Fix Release Date:
                11/Nov/19

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 5 hours, 40 minutes
                  5h 40m