Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-55072

Upgrades to Behat so it can work with different themes

XMLWordPrintable

    • MOODLE_32_STABLE
    • MOODLE_32_STABLE
    • wip-mdl-55072
    • Hide

      NOTE: Features in Snap theme might fail, please ignore them.

      Test if behat.yml is being created properly.

      NOTE:

      1. behat_theme_snap_* context should only be available in snap suite
      2. behat_theme_snap_behat_selectors should not be included in behat.yml
      1. Test 1:
        • Initialise behat without patch (admin/tool/behat/cli/init.php)
        • Initialise behat with patch (admin/tool/behat/cli/init.php)
        • Compare behat.yml for default suite section and ensure they are same.
      1. Test 2:
        • Initialise behat without patch for parallel run (admin/tool/behat/cli/init.php -j=2)
        • Initialise behat with patch for parallel run (admin/tool/behat/cli/init.php -j=2)
        • compare behat.yml for run 1 and run2, and ensure default suite section is same for both behat.yml
      1. Test 3: clone https://github.com/rajeshtaneja/snap in your theme/ directory
        • Initilise behat without patch (admin/tool/behat/cli/init.php)
        • Initilise behat with patch (admin/tool/behat/cli/init.php)
        • Compare behat.yml for above and make sure contexts and features under suites > default is same (except theme snap features)
        • Ensure you can see paths and contexts under snap suite and it only contains missing themes ^
      1. Test 4: clone https://github.com/rajeshtaneja/snap in your theme/ directory
        • Initilise behat without patch (admin/tool/behat/cli/init.php)
        • Initilise behat with patch (admin/tool/behat/cli/init.php --run-with-theme)
        • Compare behat.yml for each run and make sure contexts and features under suites > default is same (except theme snap features)
        • Ensure you can see paths and contexts under snap suite and it only contains missing themes ^ + features from ^ - features mentioned in blacklist.json
      1. Test 5: Add "grade/tests/behat/behat_grade.php" in contexts in file /theme/snap/tests/behat/blacklist.json
        • Initialise behat with patch (admin/tool/behat/cli/init.php)
        • Ensure in behat.yml behat_grade, behat_navigation and behat_auth should not exist under contexts for snap suite, and there should be behat_theme_snap_behat_navigation and behat_theme_snap_behat_auth
        • Ensure in behat.yml features blacklisted in blacklist.json should not exist under paths for snap suite
        • change contexts name from "grade/tests/behat/behat_grade.php" to "behat_grade" and ensure after initialising behat_grade is not in contexts for snap suite
        • change contexts name from "behat_grade" to "behat_grade.php" and ensure after initialising behat_grade is not in contexts for snap suite

      Test all suites are run.

      1. Test 1:
        • Initialise behat (without --run-with-theme)
        • run scenario --name="Bold some text"
        • Ensure only 1 test is executed.
      2. Test 2:
        • Initialise behat (with --run-with-theme)
        • run scenario --name="Bold some text"
        • Ensure 3 test are executed, one with core theme, one with more theme and last with snap theme
      3. Test 3:
        • Initialise behat (with --run-with-theme)
        • run scenario --name="Student does not see admin block on any page."
        • Ensure only 1 test is executed with theme snap.
      4. Test 4:
        • Drop old behat site and Initialise behat with 2 runs (-j=2)
        • run php admin/tool/behat/cli/run.php --tags=@mod_assign
        • Ensure you can only see 1 runs with default theme only.
        • Check behat.yml for both runs and ensure there is no features under path for snap or more suite.
      5. Test 5:
        • run php admin/tool/behat/cli/run.php --tags=@mod_assign --run-with-theme
        • Ensure you can only see 2 runs with default theme, more theme and then with snap theme.
        • Check behat.yml for both runs and ensure there are features under path for snap suite.
      6. Test 6:
        • run php admin/tool/behat/cli/run.php --tags=@mod_assign --run-with-theme --suite=snap
        • Ensure you can only see 2 runs with snap theme only.
        • Check behat.yml for both runs and ensure there are features under path for snap suite.
      7. Test 7:
        • run php admin/tool/behat/cli/run.php --tags=@mod_assign --suite=snap
        • Ensure you can only see 2 runs started with no scenarios.
        • Check behat.yml for both runs and ensure there is no features under path for snap suite.

      Unit tests:

      1. Run unit test admin/tool/behat/tests/manager_test.php

      Extra credit

      1. Initialise behat (without --run-with-theme)
      2. Run behat (--format=pretty --out=~/pretty.txt --format=progress --out=std)
      3. After the run ensure all features in core were executed, default and theme specific. (Ignore behat failures.)

      Important: Test above on windows.

      Show
      NOTE: Features in Snap theme might fail, please ignore them. Test if behat.yml is being created properly. NOTE: behat_theme_snap_* context should only be available in snap suite behat_theme_snap_behat_selectors should not be included in behat.yml Test 1: Initialise behat without patch (admin/tool/behat/cli/init.php) Initialise behat with patch (admin/tool/behat/cli/init.php) Compare behat.yml for default suite section and ensure they are same. Test 2: Initialise behat without patch for parallel run (admin/tool/behat/cli/init.php -j=2) Initialise behat with patch for parallel run (admin/tool/behat/cli/init.php -j=2) compare behat.yml for run 1 and run2, and ensure default suite section is same for both behat.yml Test 3: clone https://github.com/rajeshtaneja/snap in your theme/ directory Initilise behat without patch (admin/tool/behat/cli/init.php) Initilise behat with patch (admin/tool/behat/cli/init.php) Compare behat.yml for above and make sure contexts and features under suites > default is same (except theme snap features) Ensure you can see paths and contexts under snap suite and it only contains missing themes ^ Test 4: clone https://github.com/rajeshtaneja/snap in your theme/ directory Initilise behat without patch (admin/tool/behat/cli/init.php) Initilise behat with patch (admin/tool/behat/cli/init.php --run-with-theme) Compare behat.yml for each run and make sure contexts and features under suites > default is same (except theme snap features) Ensure you can see paths and contexts under snap suite and it only contains missing themes ^ + features from ^ - features mentioned in blacklist.json Test 5: Add "grade/tests/behat/behat_grade.php" in contexts in file /theme/snap/tests/behat/blacklist.json Initialise behat with patch (admin/tool/behat/cli/init.php) Ensure in behat.yml behat_grade, behat_navigation and behat_auth should not exist under contexts for snap suite, and there should be behat_theme_snap_behat_navigation and behat_theme_snap_behat_auth Ensure in behat.yml features blacklisted in blacklist.json should not exist under paths for snap suite change contexts name from "grade/tests/behat/behat_grade.php" to "behat_grade" and ensure after initialising behat_grade is not in contexts for snap suite change contexts name from "behat_grade" to "behat_grade.php" and ensure after initialising behat_grade is not in contexts for snap suite Test all suites are run. Test 1: Initialise behat (without --run-with-theme) run scenario --name="Bold some text" Ensure only 1 test is executed. Test 2: Initialise behat (with --run-with-theme) run scenario --name="Bold some text" Ensure 3 test are executed, one with core theme, one with more theme and last with snap theme Test 3: Initialise behat (with --run-with-theme) run scenario --name="Student does not see admin block on any page." Ensure only 1 test is executed with theme snap. Test 4: Drop old behat site and Initialise behat with 2 runs (-j=2) run php admin/tool/behat/cli/run.php --tags=@mod_assign Ensure you can only see 1 runs with default theme only. Check behat.yml for both runs and ensure there is no features under path for snap or more suite. Test 5: run php admin/tool/behat/cli/run.php --tags=@mod_assign --run-with-theme Ensure you can only see 2 runs with default theme, more theme and then with snap theme. Check behat.yml for both runs and ensure there are features under path for snap suite. Test 6: run php admin/tool/behat/cli/run.php --tags=@mod_assign --run-with-theme --suite=snap Ensure you can only see 2 runs with snap theme only. Check behat.yml for both runs and ensure there are features under path for snap suite. Test 7: run php admin/tool/behat/cli/run.php --tags=@mod_assign --suite=snap Ensure you can only see 2 runs started with no scenarios. Check behat.yml for both runs and ensure there is no features under path for snap suite. Unit tests: Run unit test admin/tool/behat/tests/manager_test.php Extra credit Initialise behat (without --run-with-theme) Run behat (--format=pretty --out=~/pretty.txt --format=progress --out=std) After the run ensure all features in core were executed, default and theme specific. (Ignore behat failures.) Important: Test above on windows.

      Change behat infrastructure so steps can be overridden by themes. Should allow whole suite to pass on different themes even with different navigation etc.

        1. behat_rubric_run1.yml
          11 kB
        2. behat_rubric_run2.yml
          8 kB

            rajeshtaneja Rajesh Taneja
            damyon Damyon Wiese
            Damyon Wiese Damyon Wiese
            David Monllaó David Monllaó
            Jun Pataleta Jun Pataleta
            Votes:
            3 Vote for this issue
            Watchers:
            11 Start watching this issue

              Created:
              Updated:
              Resolved:

                Error rendering 'clockify-timesheets-time-tracking-reports:timer-sidebar'. Please contact your Jira administrators.