Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-46891

Upgrade to behat 3.x

XMLWordPrintable

    • MOODLE_27_STABLE, MOODLE_31_STABLE
    • MOODLE_31_STABLE
    • wip-mdl-46891
    • Difficult
    • Hide

      Pre-test: make sure that git grep MDL-46891 does not return anything

      FYI: For testing please delete composer.lock and vendor directory before initializing behat.

      1. Basic test
      1. Run behat suite with single run 

         vendor/bin/behat --config path_to_behat/behat.yml

      2. Run behat suite with multiple runs (3) 

        php admin/tool/behat/cli/init.php -j=3
        		 
         php admin/tool/behat/cli/run.php

      2. Test Formatters
      1. moodle_progress Run behat with --format moodle_progress and ensure you see moodle branch information before dots start.
      2. moodle_list: List all scenarios
      3. moodle_stepcount: List step cont in all features.
      4. moodle_screenshot: Take screenshot and core dump of each step. With following options you can dump either or both.
        • --format-settings '{"formats": "image"}' : will dump image only
        • --format-settings '{"formats": "html"}': will dump html only.
        • --format-settings '{"formats": "html,image"}': will dump both.
        • --format-settings '{"formats": "html", "dir_permissions": 0777}' will set proper directory permssions
        • if used --out path_to_save then screenshots should be saved at specified path
      3. Test multiple formatters
      1. to use multiple formatters use --format moodle_progress --out std --format moodle_screenshot --out /home/dump
      4. Check exceptions are caught in step and chained steps.
      1. Run attached file and ensure you see failure 4 fails.
      2. Modify view.php with following patch and ensure you see 6 fails. 

        diff --git a/course/view.php b/course/view.php
        		 
        index e15a022..0b5a9bf 100644
        		 
        --- a/course/view.php
        		 
        +++ b/course/view.php
        		 
        @@ -145,6 +145,7 @@
        		 
             }
        		 
             if ($PAGE->user_allowed_editing()) {
        		 
                 if (($edit == 1) and confirm_sesskey()) {
        		 
        +            throw new moodle_exception("Testing behat3");
        		 
                     $USER->editing = 1;
        		 
                     // Redirect to site root if Editing is toggled on frontpage
        		 
                     if ($course->id == SITEID) {

      5. Test $CFG->behat_profiles and $CFG->behat_config
      1. Set following in your config.php and ensure phantomjs and chrome profiles are set in behat.yml
      2. Start runs with both --profile=phantomjs and --profile=chrome 

        $CFG->behat_profiles = array(
        		 
            'phantomjs' => array(
        		 
                'browser' => 'phantomjs',
        		 
                'tags' => '~@_file_upload&&~@_alert&&~@_bug_phantomjs',
        		 
                'wd_host' => 'http://127.0.0.1:4443/wd/hub',
        		 
                'capabilities' => array(
        		 
                    'platform' => 'Linux',
        		 
                    'version' => 2.1
        		 
                )
        		 
            )
        		 
        );
        		 
         
        		 
        $CFG->behat_config = array(
        		 
            'chrome' => array(
        		 
               'extensions' => array(
        		 
                   'Behat\MinkExtension\Extension' => array(
        		 
                       'selenium2' => array(
        		 
                           'browser' => 'chrome',
        		 
                       )
        		 
                   )
        		 
               )
        		 
            )
        		 
        );

      6. Exit code of parallel run.
      1. Initialise behat with 3 runs. (php admin/tool/behat/cli/init.php -j=3)
      2. Modify behat.yml and leave only 3 feature files.
      3. Modify 1st and 3rd feature file so they fail.
      4. Run parallel behat and check exit code. It should be 5 (php admin/tool/behat/cli/run.php) (echo $?)
      5. Modify 2nd feature file, so it fails
      6. Run parallel behat and check exit code is 7 (php admin/tool/behat/cli/run.php) (echo $?)
      7. Check parallel run profile can be modified for each run.
      1. Add profile in config.php firefox1, firefox2, firefox3
      2. Initialise behat
      3. run php admin/tool/behat/cli/run.php --profile=firefox {runprocess} --replace="{runprocess}

        "

      4. Make sure you don't get any error.
      Show
      Pre-test: make sure that git grep MDL-46891 does not return anything FYI: For testing please delete composer.lock and vendor directory before initializing behat. 1. Basic test Run behat suite with single run vendor/bin/behat --config path_to_behat/behat.yml Run behat suite with multiple runs (3) php admin/tool/behat/cli/init.php -j=3 php admin/tool/behat/cli/run.php 2. Test Formatters moodle_progress Run behat with --format moodle_progress and ensure you see moodle branch information before dots start. moodle_list : List all scenarios moodle_stepcount : List step cont in all features. moodle_screenshot : Take screenshot and core dump of each step. With following options you can dump either or both. --format-settings '{"formats": "image"}' : will dump image only --format-settings '{"formats": "html"}': will dump html only. --format-settings '{"formats": "html,image"}': will dump both. --format-settings '{"formats": "html", "dir_permissions": 0777}' will set proper directory permssions if used --out path_to_save then screenshots should be saved at specified path 3. Test multiple formatters to use multiple formatters use --format moodle_progress --out std --format moodle_screenshot --out /home/dump 4. Check exceptions are caught in step and chained steps. Run attached file and ensure you see failure 4 fails. Modify view.php with following patch and ensure you see 6 fails. diff --git a/course/view.php b/course/view.php index e15a022..0b5a9bf 100644 --- a/course/view.php +++ b/course/view.php @@ -145,6 +145,7 @@ } if ($PAGE->user_allowed_editing()) { if (($edit == 1) and confirm_sesskey()) { + throw new moodle_exception("Testing behat3"); $USER->editing = 1; // Redirect to site root if Editing is toggled on frontpage if ($course->id == SITEID) { 5. Test $CFG->behat_profiles and $CFG->behat_config Set following in your config.php and ensure phantomjs and chrome profiles are set in behat.yml Start runs with both --profile=phantomjs and --profile=chrome $CFG->behat_profiles = array( 'phantomjs' => array( 'browser' => 'phantomjs', 'tags' => '~@_file_upload&&~@_alert&&~@_bug_phantomjs', 'wd_host' => 'http://127.0.0.1:4443/wd/hub', 'capabilities' => array( 'platform' => 'Linux', 'version' => 2.1 ) ) );   $CFG->behat_config = array( 'chrome' => array( 'extensions' => array( 'Behat\MinkExtension\Extension' => array( 'selenium2' => array( 'browser' => 'chrome', ) ) ) ) ); 6. Exit code of parallel run. Initialise behat with 3 runs. (php admin/tool/behat/cli/init.php -j=3) Modify behat.yml and leave only 3 feature files. Modify 1st and 3rd feature file so they fail. Run parallel behat and check exit code. It should be 5 (php admin/tool/behat/cli/run.php) (echo $?) Modify 2nd feature file, so it fails Run parallel behat and check exit code is 7 (php admin/tool/behat/cli/run.php) (echo $?) 7. Check parallel run profile can be modified for each run. Add profile in config.php firefox1, firefox2, firefox3 Initialise behat run php admin/tool/behat/cli/run.php --profile=firefox {runprocess} --replace="{runprocess} " Make sure you don't get any error.
    • Team Beards Sprint 2
    • Small

      I don't think there is an issue tracking this, but behat 3 has been out for a while now and we should make sure we do not get left too far behind:

      https://github.com/Behat/Behat/releases

      Unfortunately there doesn't seem to be a simple guide for the migration:
      https://github.com/Behat/docs/issues/35

        1. failingbehat.feature
          1 kB
          Rajesh Taneja

            rajeshtaneja Rajesh Taneja
            poltawski Dan Poltawski
            David Monllaó David Monllaó
            Eloy Lafuente (stronk7) Eloy Lafuente (stronk7)
            John Okely John Okely
            Votes:
            0 Vote for this issue
            Watchers:
            15 Start watching this issue

              Created:
              Updated:
              Resolved:

                Error rendering 'clockify-timesheets-time-tracking-reports:timer-sidebar'. Please contact your Jira administrators.