Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-75081

Checking table values in Behat is broken

    XMLWordPrintable

Details

    • MOODLE_400_STABLE, MOODLE_401_STABLE, MOODLE_402_STABLE, MOODLE_403_STABLE
    • MOODLE_401_STABLE, MOODLE_402_STABLE, MOODLE_403_STABLE
    • MDL-75081-402
    • MDL-75081-403
    • Hide
      1. Automated tests will confirm everything is working

      Manual test for false positives

      We expect 4 of the 8 scenarios in the attached file to fail (on master/main only 2 of them fail)

      1. Download tables.feature to <DIRROOT>/admin/tests/behat/
      2. Initialize Behat locally, e.g.

        $ php admin/tool/behat/cli/init.php
        

      3. Execute the tests, e.g.

        $ php admin/tool/behat/cli/run.php --name="MDL-75081"
        

      4. Confirm you see:

        8 scenarios (4 passed, 4 failed)
        

      5. Confirm the failing scenarios are:
        • 2. Check single column exists (expect fail)
        • 4. Check multiple columns exist (expect fail)
        • 6. Check single column doesn't exist (expect fail)
        • 8. Check multiple columns don't exist (expect fail)
      Show
      Automated tests will confirm everything is working Manual test for false positives We expect 4 of the 8 scenarios in the attached file to fail (on master/main only 2 of them fail) Download tables.feature to <DIRROOT>/admin/tests/behat/ Initialize Behat locally, e.g. $ php admin/tool/behat/cli/init.php Execute the tests, e.g. $ php admin/tool/behat/cli/run.php --name="MDL-75081" Confirm you see: 8 scenarios (4 passed, 4 failed) Confirm the failing scenarios are: 2. Check single column exists (expect fail) 4. Check multiple columns exist (expect fail) 6. Check single column doesn't exist (expect fail) 8. Check multiple columns don't exist (expect fail)

    Description

      The Behat step to check for values in tables is broken and does not give proper results. I'm not sure what's the actual problem, but even something like this passes instead of failing:

      Then the following should exist in the "doesnotexist" table:
            | Name             |
            | thisdoesnotexist |
      

      And then some tests that should be passing fail instead, so I don't think this step is doing its job and it may be dangerous to use until it's fixed.

      Here's a reproduction with some botched scenarios: https://github.com/NoelDeMartin/moodle-local_sandbox/blob/MDL-75081/tests/behat/sandbox.feature

      Attachments

        Issue Links

          Activity

            People

              pholden Paul Holden
              noeldemartin Noel De Martin
              Mathew May Mathew May
              Jun Pataleta Jun Pataleta
              Kim Jared Lucas Kim Jared Lucas
              Votes:
              1 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 1 day, 3 hours, 17 minutes
                  1d 3h 17m

                  Clockify

                    Error rendering 'clockify-timesheets-time-tracking-reports:timer-sidebar'. Please contact your Jira administrators.