Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-68525

Behat broken since MDL-67317 + Chrome 81 using $CFG->behat_profiles as in the docs

    XMLWordPrintable

    Details

    • Testing Instructions:
      Hide

      requirements

      1. Ability to install multiple chrome and chromedriver versions
        1. chrome: https://www.slimjet.com/chrome/google-chrome-old-version.php
        2. chromedriver: https://chromedriver.chromium.org/downloads
      2. To be tested in 37, 38 and master

      normal testing

      1. Install Chrome (and chromedriver) 79.
      2. Install a new site
      3. Edit config.php and add these lines, anywhere before the last lines:

        $CFG->behat_profiles = [
            'seleniumchrome' => [
                'browser' => 'chrome',
                'extensions' => [
                    'Behat\MinkExtension' => [
                        'selenium2' => [
                            'browser' => 'chrome',
                        ]
                    ]
                ]
            ]
        ];
        

      4. Init behat with php admin/tools/behat/cli/init -a
      5. Run behat with the command suggested by init adding this --profile seleniumchrome --tags '@mod_page'
      6. Verify that Chrome is opened normally and tests run.
      7. Wait a little bit until the run ends (a couple of mins).
      8. Verify that the job ends ok, with 6 scenarios, all passed.
      9. Repeat the steps above with Chrome (and chromedriver) 81.

      new chromeOptions/args passed work as expected.

      (instructions borrowed from MDL-64967)
      You must have a developer setup with Behat working, including Behat mobile app testing, with a Chrome browser profile - see https://docs.moodle.org/dev/Acceptance_testing_for_the_mobile_app

      1. There are 2 test files attached to this issue, mdl68525.feature and mdl68525.css. Download both these files.
      2. Install Chrome (and chromedriver) 79.
      3. Install a new site
      4. Edit config.php and add these lines, anywhere before the last lines:

        $CFG->behat_profiles = [
            'seleniumchrome' => [
                'browser' => 'chrome',
                'extensions' => [
                    'Behat\MinkExtension' => [
                        'selenium2' => [
                            'browser' => 'chrome',
                        ]
                    ]
                ]
            ]
        ];
        

      5. Put the files into lib/tests/behat folder.
      6. Edit the .feature file - there is a line about mobilecssurl. Change this line so that the URL is correct for your server.
      7. Run Behat init (php admin/tool/behat/cli/init.php)
      8. Edit the reported location for the behat.yml file, and open the file
      9. In the file, scroll down until you find the "seleniumchrome" profile area (beginning 'chrome:' at the start of a line).
      10. Verify that it includes, under 'extra_capabilities / chromeOptions / args the following lines:

        - '--unlimited-storage'
        - '--disable-web-security'
        

      11. Run behat with the command suggested by init adding this --profile seleniumchrome --tags '@mdl68525'
      12. Verify that the test passes (The test checks that the CSS file has taken effect.)
      13. Verify that the header stripe has a pink-black gradient.
      14. Repeat the steps above with Chrome (and chromedriver) 81.
      Show
      requirements Ability to install multiple chrome and chromedriver versions chrome: https://www.slimjet.com/chrome/google-chrome-old-version.php chromedriver: https://chromedriver.chromium.org/downloads To be tested in 37, 38 and master normal testing Install Chrome (and chromedriver) 79. Install a new site Edit config.php and add these lines, anywhere before the last lines: $CFG->behat_profiles = [ 'seleniumchrome' => [ 'browser' => 'chrome', 'extensions' => [ 'Behat\MinkExtension' => [ 'selenium2' => [ 'browser' => 'chrome', ] ] ] ] ]; Init behat with php admin/tools/behat/cli/init -a Run behat with the command suggested by init adding this --profile seleniumchrome --tags '@mod_page' Verify that Chrome is opened normally and tests run. Wait a little bit until the run ends (a couple of mins). Verify that the job ends ok, with 6 scenarios, all passed. Repeat the steps above with Chrome (and chromedriver) 81. new chromeOptions/args passed work as expected. (instructions borrowed from MDL-64967 ) You must have a developer setup with Behat working, including Behat mobile app testing, with a Chrome browser profile - see https://docs.moodle.org/dev/Acceptance_testing_for_the_mobile_app There are 2 test files attached to this issue, mdl68525.feature and mdl68525.css . Download both these files. Install Chrome (and chromedriver) 79. Install a new site Edit config.php and add these lines, anywhere before the last lines: $CFG->behat_profiles = [ 'seleniumchrome' => [ 'browser' => 'chrome', 'extensions' => [ 'Behat\MinkExtension' => [ 'selenium2' => [ 'browser' => 'chrome', ] ] ] ] ]; Put the files into lib/tests/behat folder. Edit the .feature file - there is a line about mobilecssurl. Change this line so that the URL is correct for your server. Run Behat init (php admin/tool/behat/cli/init.php) Edit the reported location for the behat.yml file, and open the file In the file, scroll down until you find the "seleniumchrome" profile area (beginning 'chrome:' at the start of a line). Verify that it includes, under 'extra_capabilities / chromeOptions / args the following lines: - '--unlimited-storage' - '--disable-web-security' Run behat with the command suggested by init adding this --profile seleniumchrome --tags '@mdl68525' Verify that the test passes (The test checks that the CSS file has taken effect.) Verify that the header stripe has a pink-black gradient. Repeat the steps above with Chrome (and chromedriver) 81.
    • Workaround:
      Hide

      Change your config.php to use $CFG->behat_config, as at https://pastebin.com/Jz6FGUHf.

      Show
      Change your config.php to use $CFG->behat_config, as at https://pastebin.com/Jz6FGUHf .
    • Affected Branches:
      MOODLE_38_STABLE
    • Fixed Branches:
      MOODLE_37_STABLE, MOODLE_38_STABLE
    • Pull 3.7 Branch:
    • Pull 3.8 Branch:
    • Pull Master Branch:
      MDL-68525-master

      Description

      To reproduce:

      1. On a Windows computer.
      2. Check out latest master.
      3. Init Behat following the instructions on https://docs.moodle.org/dev/Running_acceptance_test. In particular copy/paste the $CFG->behat_profiles = [...]; into your config,php.
      4. Run any Behat tests tagged @javascript.

      Expected result: it works.

      Actual result: it fails with the "The base URL (http://localhost/moodle) is not a behat test site."

      To verify it was the recent change that broke this.:

      1. git checkout 788dfb9c7dad27824ee52
      2. Re-init Behat. (php admin/tool/behat/cli/init.php)
      3. Run any Behat tests tagged @javascript.

      then it works.

      If, instead, you change your config.php to set $CFG->behat_config (e.g. as at https://pastebin.com/Jz6FGUHf) then it works.

      TODO: Don't forget to review and unhold (currently held @ integration) MDL-68445, depending of the changes here it can be unlocked. If solution here involves downgrading or affects to that issue in any way... then we need to send it back to dev.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              stronk7 Eloy Lafuente (stronk7)
              Reporter:
              timhunt Tim Hunt
              Peer reviewer:
              Tim Hunt
              Integrator:
              Andrew Nicols
              Tester:
              Andrew Nicols
              Participants:
              Component watchers:
              Andrew Nicols, Jun Pataleta, Michael Hawkins, Shamim Rezaie, Simey Lameze
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:
                Fix Release Date:
                11/May/20

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 6 hours, 49 minutes
                  6h 49m