Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-75081

Checking table values in Behat is broken

XMLWordPrintable

    • MOODLE_400_STABLE, MOODLE_401_STABLE, MOODLE_402_STABLE, MOODLE_403_STABLE
    • MOODLE_401_STABLE, MOODLE_402_STABLE, MOODLE_403_STABLE
    • MDL-75081-402
    • MDL-75081-403
    • Hide
      1. Automated tests will confirm everything is working

      Manual test for false positives

      We expect 4 of the 8 scenarios in the attached file to fail (on master/main only 2 of them fail)

      1. Download tables.feature to <DIRROOT>/admin/tests/behat/
      2. Initialize Behat locally, e.g. 

        $ php admin/tool/behat/cli/init.php

      3. Execute the tests, e.g. 

        $ php admin/tool/behat/cli/run.php --name="MDL-75081"

      4. Confirm you see: 

        8 scenarios (4 passed, 4 failed)

      5. Confirm the failing scenarios are:
        • 2. Check single column exists (expect fail)
        • 4. Check multiple columns exist (expect fail)
        • 6. Check single column doesn't exist (expect fail)
        • 8. Check multiple columns don't exist (expect fail)
      Show
      Automated tests will confirm everything is working Manual test for false positives We expect 4 of the 8 scenarios in the attached file to fail (on master/main only 2 of them fail) Download tables.feature to <DIRROOT>/admin/tests/behat/ Initialize Behat locally, e.g. $ php admin/tool/behat/cli/init.php Execute the tests, e.g. $ php admin/tool/behat/cli/run.php --name="MDL-75081" Confirm you see: 8 scenarios (4 passed, 4 failed) Confirm the failing scenarios are: 2. Check single column exists (expect fail) 4. Check multiple columns exist (expect fail) 6. Check single column doesn't exist (expect fail) 8. Check multiple columns don't exist (expect fail)

      The Behat step to check for values in tables is broken and does not give proper results. I'm not sure what's the actual problem, but even something like this passes instead of failing:

      Then the following should exist in the "doesnotexist" table:
      		 
            | Name             |
      		 
            | thisdoesnotexist |

      And then some tests that should be passing fail instead, so I don't think this step is doing its job and it may be dangerous to use until it's fixed.

      Here's a reproduction with some botched scenarios: https://github.com/NoelDeMartin/moodle-local_sandbox/blob/MDL-75081/tests/behat/sandbox.feature

        1. (1) Passed -- (402)MDL-75081.png
          (1) Passed -- (402)MDL-75081.png
          179 kB
        2. (1) Passed -- (Main)MDL-75081.png
          (1) Passed -- (Main)MDL-75081.png
          178 kB
        3. tables.feature
          2 kB

            pholden Paul Holden
            noeldemartin Noel De Martin
            Mathew May Mathew May
            Jun Pataleta Jun Pataleta
            Kim Jared Lucas Kim Jared Lucas
            Votes:
            1 Vote for this issue
            Watchers:
            14 Start watching this issue

              Created:
              Updated:
              Resolved:

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0 minutes
                0m
                Logged:
                Time Spent - 1 day, 3 hours, 17 minutes
                1d 3h 17m

                  Error rendering 'clockify-timesheets-time-tracking-reports:timer-sidebar'. Please contact your Jira administrators.