Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-46891

Upgrade to behat 3.x

    XMLWordPrintable

    Details

    • Testing Instructions:
      Hide

      Pre-test: make sure that git grep MDL-46891 does not return anything

      FYI: For testing please delete composer.lock and vendor directory before initializing behat.

      1. Basic test
      1. Run behat suite with single run

         vendor/bin/behat --config path_to_behat/behat.yml

      2. Run behat suite with multiple runs (3)

        php admin/tool/behat/cli/init.php -j=3
         php admin/tool/behat/cli/run.php
        

      2. Test Formatters
      1. moodle_progress Run behat with --format moodle_progress and ensure you see moodle branch information before dots start.
      2. moodle_list: List all scenarios
      3. moodle_stepcount: List step cont in all features.
      4. moodle_screenshot: Take screenshot and core dump of each step. With following options you can dump either or both.
        • --format-settings '{"formats": "image"}' : will dump image only
        • --format-settings '{"formats": "html"}': will dump html only.
        • --format-settings '{"formats": "html,image"}': will dump both.
        • --format-settings '{"formats": "html", "dir_permissions": 0777}' will set proper directory permssions
        • if used --out path_to_save then screenshots should be saved at specified path
      3. Test multiple formatters
      1. to use multiple formatters use --format moodle_progress --out std --format moodle_screenshot --out /home/dump
      4. Check exceptions are caught in step and chained steps.
      1. Run attached file and ensure you see failure 4 fails.
      2. Modify view.php with following patch and ensure you see 6 fails.

        diff --git a/course/view.php b/course/view.php
        index e15a022..0b5a9bf 100644
        --- a/course/view.php
        +++ b/course/view.php
        @@ -145,6 +145,7 @@
             }
             if ($PAGE->user_allowed_editing()) {
                 if (($edit == 1) and confirm_sesskey()) {
        +            throw new moodle_exception("Testing behat3");
                     $USER->editing = 1;
                     // Redirect to site root if Editing is toggled on frontpage
                     if ($course->id == SITEID) {
        

      5. Test $CFG->behat_profiles and $CFG->behat_config
      1. Set following in your config.php and ensure phantomjs and chrome profiles are set in behat.yml
      2. Start runs with both --profile=phantomjs and --profile=chrome

        $CFG->behat_profiles = array(
            'phantomjs' => array(
                'browser' => 'phantomjs',
                'tags' => '~@_file_upload&&~@_alert&&~@_bug_phantomjs',
                'wd_host' => 'http://127.0.0.1:4443/wd/hub',
                'capabilities' => array(
                    'platform' => 'Linux',
                    'version' => 2.1
                )
            )
        );
         
        $CFG->behat_config = array(
            'chrome' => array(
               'extensions' => array(
                   'Behat\MinkExtension\Extension' => array(
                       'selenium2' => array(
                           'browser' => 'chrome',
                       )
                   )
               )
            )
        );
        

      6. Exit code of parallel run.
      1. Initialise behat with 3 runs. (php admin/tool/behat/cli/init.php -j=3)
      2. Modify behat.yml and leave only 3 feature files.
      3. Modify 1st and 3rd feature file so they fail.
      4. Run parallel behat and check exit code. It should be 5 (php admin/tool/behat/cli/run.php) (echo $?)
      5. Modify 2nd feature file, so it fails
      6. Run parallel behat and check exit code is 7 (php admin/tool/behat/cli/run.php) (echo $?)
      7. Check parallel run profile can be modified for each run.
      1. Add profile in config.php firefox1, firefox2, firefox3
      2. Initialise behat
      3. run php admin/tool/behat/cli/run.php --profile=firefox {runprocess} --replace="{runprocess}

        "

      4. Make sure you don't get any error.
      Show
      Pre-test: make sure that git grep MDL-46891 does not return anything FYI: For testing please delete composer.lock and vendor directory before initializing behat. 1. Basic test Run behat suite with single run vendor/bin/behat --config path_to_behat/behat.yml Run behat suite with multiple runs (3) php admin/tool/behat/cli/init.php -j=3 php admin/tool/behat/cli/run.php 2. Test Formatters moodle_progress Run behat with --format moodle_progress and ensure you see moodle branch information before dots start. moodle_list : List all scenarios moodle_stepcount : List step cont in all features. moodle_screenshot : Take screenshot and core dump of each step. With following options you can dump either or both. --format-settings '{"formats": "image"}' : will dump image only --format-settings '{"formats": "html"}': will dump html only. --format-settings '{"formats": "html,image"}': will dump both. --format-settings '{"formats": "html", "dir_permissions": 0777}' will set proper directory permssions if used --out path_to_save then screenshots should be saved at specified path 3. Test multiple formatters to use multiple formatters use --format moodle_progress --out std --format moodle_screenshot --out /home/dump 4. Check exceptions are caught in step and chained steps. Run attached file and ensure you see failure 4 fails. Modify view.php with following patch and ensure you see 6 fails. diff --git a/course/view.php b/course/view.php index e15a022..0b5a9bf 100644 --- a/course/view.php +++ b/course/view.php @@ -145,6 +145,7 @@ } if ($PAGE->user_allowed_editing()) { if (($edit == 1) and confirm_sesskey()) { + throw new moodle_exception("Testing behat3"); $USER->editing = 1; // Redirect to site root if Editing is toggled on frontpage if ($course->id == SITEID) { 5. Test $CFG->behat_profiles and $CFG->behat_config Set following in your config.php and ensure phantomjs and chrome profiles are set in behat.yml Start runs with both --profile=phantomjs and --profile=chrome $CFG->behat_profiles = array( 'phantomjs' => array( 'browser' => 'phantomjs', 'tags' => '~@_file_upload&&~@_alert&&~@_bug_phantomjs', 'wd_host' => 'http://127.0.0.1:4443/wd/hub', 'capabilities' => array( 'platform' => 'Linux', 'version' => 2.1 ) ) );   $CFG->behat_config = array( 'chrome' => array( 'extensions' => array( 'Behat\MinkExtension\Extension' => array( 'selenium2' => array( 'browser' => 'chrome', ) ) ) ) ); 6. Exit code of parallel run. Initialise behat with 3 runs. (php admin/tool/behat/cli/init.php -j=3) Modify behat.yml and leave only 3 feature files. Modify 1st and 3rd feature file so they fail. Run parallel behat and check exit code. It should be 5 (php admin/tool/behat/cli/run.php) (echo $?) Modify 2nd feature file, so it fails Run parallel behat and check exit code is 7 (php admin/tool/behat/cli/run.php) (echo $?) 7. Check parallel run profile can be modified for each run. Add profile in config.php firefox1, firefox2, firefox3 Initialise behat run php admin/tool/behat/cli/run.php --profile=firefox {runprocess} --replace="{runprocess} " Make sure you don't get any error.
    • Difficulty:
      Difficult
    • Affected Branches:
      MOODLE_27_STABLE, MOODLE_31_STABLE
    • Fixed Branches:
      MOODLE_31_STABLE
    • Pull Master Branch:
      wip-mdl-46891
    • Sprint:
      Team Beards Sprint 2
    • Issue size:
      Small

      Description

      I don't think there is an issue tracking this, but behat 3 has been out for a while now and we should make sure we do not get left too far behind:

      https://github.com/Behat/Behat/releases

      Unfortunately there doesn't seem to be a simple guide for the migration:
      https://github.com/Behat/docs/issues/35

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                15 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:
                  Fix Release Date:
                  23/May/16