Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-46891

Upgrade to behat 3.x

    XMLWordPrintable

Details

    • MOODLE_27_STABLE, MOODLE_31_STABLE
    • MOODLE_31_STABLE
    • wip-mdl-46891
    • Difficult
    • Hide

      Pre-test: make sure that git grep MDL-46891 does not return anything

      FYI: For testing please delete composer.lock and vendor directory before initializing behat.

      1. Basic test
      1. Run behat suite with single run

         vendor/bin/behat --config path_to_behat/behat.yml

      2. Run behat suite with multiple runs (3)

        php admin/tool/behat/cli/init.php -j=3
         php admin/tool/behat/cli/run.php
        

      2. Test Formatters
      1. moodle_progress Run behat with --format moodle_progress and ensure you see moodle branch information before dots start.
      2. moodle_list: List all scenarios
      3. moodle_stepcount: List step cont in all features.
      4. moodle_screenshot: Take screenshot and core dump of each step. With following options you can dump either or both.
        • --format-settings '{"formats": "image"}' : will dump image only
        • --format-settings '{"formats": "html"}': will dump html only.
        • --format-settings '{"formats": "html,image"}': will dump both.
        • --format-settings '{"formats": "html", "dir_permissions": 0777}' will set proper directory permssions
        • if used --out path_to_save then screenshots should be saved at specified path
      3. Test multiple formatters
      1. to use multiple formatters use --format moodle_progress --out std --format moodle_screenshot --out /home/dump
      4. Check exceptions are caught in step and chained steps.
      1. Run attached file and ensure you see failure 4 fails.
      2. Modify view.php with following patch and ensure you see 6 fails.

        diff --git a/course/view.php b/course/view.php
        index e15a022..0b5a9bf 100644
        --- a/course/view.php
        +++ b/course/view.php
        @@ -145,6 +145,7 @@
             }
             if ($PAGE->user_allowed_editing()) {
                 if (($edit == 1) and confirm_sesskey()) {
        +            throw new moodle_exception("Testing behat3");
                     $USER->editing = 1;
                     // Redirect to site root if Editing is toggled on frontpage
                     if ($course->id == SITEID) {
        

      5. Test $CFG->behat_profiles and $CFG->behat_config
      1. Set following in your config.php and ensure phantomjs and chrome profiles are set in behat.yml
      2. Start runs with both --profile=phantomjs and --profile=chrome

        $CFG->behat_profiles = array(
            'phantomjs' => array(
                'browser' => 'phantomjs',
                'tags' => '~@_file_upload&&~@_alert&&~@_bug_phantomjs',
                'wd_host' => 'http://127.0.0.1:4443/wd/hub',
                'capabilities' => array(
                    'platform' => 'Linux',
                    'version' => 2.1
                )
            )
        );
         
        $CFG->behat_config = array(
            'chrome' => array(
               'extensions' => array(
                   'Behat\MinkExtension\Extension' => array(
                       'selenium2' => array(
                           'browser' => 'chrome',
                       )
                   )
               )
            )
        );
        

      6. Exit code of parallel run.
      1. Initialise behat with 3 runs. (php admin/tool/behat/cli/init.php -j=3)
      2. Modify behat.yml and leave only 3 feature files.
      3. Modify 1st and 3rd feature file so they fail.
      4. Run parallel behat and check exit code. It should be 5 (php admin/tool/behat/cli/run.php) (echo $?)
      5. Modify 2nd feature file, so it fails
      6. Run parallel behat and check exit code is 7 (php admin/tool/behat/cli/run.php) (echo $?)
      7. Check parallel run profile can be modified for each run.
      1. Add profile in config.php firefox1, firefox2, firefox3
      2. Initialise behat
      3. run php admin/tool/behat/cli/run.php --profile=firefox {runprocess} --replace="{runprocess}

        "

      4. Make sure you don't get any error.
      Show
      Pre-test: make sure that git grep MDL-46891 does not return anything FYI: For testing please delete composer.lock and vendor directory before initializing behat. 1. Basic test Run behat suite with single run vendor/bin/behat --config path_to_behat/behat.yml Run behat suite with multiple runs (3) php admin/tool/behat/cli/init.php -j=3 php admin/tool/behat/cli/run.php 2. Test Formatters moodle_progress Run behat with --format moodle_progress and ensure you see moodle branch information before dots start. moodle_list : List all scenarios moodle_stepcount : List step cont in all features. moodle_screenshot : Take screenshot and core dump of each step. With following options you can dump either or both. --format-settings '{"formats": "image"}' : will dump image only --format-settings '{"formats": "html"}': will dump html only. --format-settings '{"formats": "html,image"}': will dump both. --format-settings '{"formats": "html", "dir_permissions": 0777}' will set proper directory permssions if used --out path_to_save then screenshots should be saved at specified path 3. Test multiple formatters to use multiple formatters use --format moodle_progress --out std --format moodle_screenshot --out /home/dump 4. Check exceptions are caught in step and chained steps. Run attached file and ensure you see failure 4 fails. Modify view.php with following patch and ensure you see 6 fails. diff --git a/course/view.php b/course/view.php index e15a022..0b5a9bf 100644 --- a/course/view.php +++ b/course/view.php @@ -145,6 +145,7 @@ } if ($PAGE->user_allowed_editing()) { if (($edit == 1) and confirm_sesskey()) { + throw new moodle_exception("Testing behat3"); $USER->editing = 1; // Redirect to site root if Editing is toggled on frontpage if ($course->id == SITEID) { 5. Test $CFG->behat_profiles and $CFG->behat_config Set following in your config.php and ensure phantomjs and chrome profiles are set in behat.yml Start runs with both --profile=phantomjs and --profile=chrome $CFG->behat_profiles = array( 'phantomjs' => array( 'browser' => 'phantomjs', 'tags' => '~@_file_upload&&~@_alert&&~@_bug_phantomjs', 'wd_host' => 'http://127.0.0.1:4443/wd/hub', 'capabilities' => array( 'platform' => 'Linux', 'version' => 2.1 ) ) );   $CFG->behat_config = array( 'chrome' => array( 'extensions' => array( 'Behat\MinkExtension\Extension' => array( 'selenium2' => array( 'browser' => 'chrome', ) ) ) ) ); 6. Exit code of parallel run. Initialise behat with 3 runs. (php admin/tool/behat/cli/init.php -j=3) Modify behat.yml and leave only 3 feature files. Modify 1st and 3rd feature file so they fail. Run parallel behat and check exit code. It should be 5 (php admin/tool/behat/cli/run.php) (echo $?) Modify 2nd feature file, so it fails Run parallel behat and check exit code is 7 (php admin/tool/behat/cli/run.php) (echo $?) 7. Check parallel run profile can be modified for each run. Add profile in config.php firefox1, firefox2, firefox3 Initialise behat run php admin/tool/behat/cli/run.php --profile=firefox {runprocess} --replace="{runprocess} " Make sure you don't get any error.
    • Team Beards Sprint 2
    • Small

    Description

      I don't think there is an issue tracking this, but behat 3 has been out for a while now and we should make sure we do not get left too far behind:

      https://github.com/Behat/Behat/releases

      Unfortunately there doesn't seem to be a simple guide for the migration:
      https://github.com/Behat/docs/issues/35

      Attachments

        Issue Links

          Activity

            People

              rajeshtaneja Rajesh Taneja
              poltawski Dan Poltawski
              David Monllaó David Monllaó
              Eloy Lafuente (stronk7) Eloy Lafuente (stronk7)
              John Okely John Okely
              Andrew Lyons, Huong Nguyen, Jun Pataleta, Michael Hawkins, Shamim Rezaie, Simey Lameze
              Votes:
              0 Vote for this issue
              Watchers:
              15 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:
                23/May/16