Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-41592

Make Behat more flexible about the wwwroot it uses to access the test site

    XMLWordPrintable

    Details

    • Testing Instructions:
      Hide

      We should test that all that combinations are working

      Note that as "production site" I mean the normal environment, and as "test site" the one titled as "Acceptance test site".

      Test 0, install behat

      1. Install behat or util.php --drop & util.php --install & util.php --enable if it was already installed
      2. It SHOULD finish as expected

      Test 1, switch completely works as expected

      1. Set this in your config.php, ensure you don't have the same vars defined below/above in your config file:
        $CFG->wwwroot = 'http://localhost/path/to/your/site';
        $CFG->behat_switchcompletely = true;
        //$CFG->behat_wwwroot = http://YOURIP/path/to/your/site
      2. Go to http://localhost/path/to/your/site
      3. You SHOULD see your test site and the page SHOULD be styled as usual
      4. Follow Log in link
      5. You SHOULD be redirected to the login page and the page SHOULD be styled as usual
      6. Go to http://YOURIP/path/to/your/site
      7. You SHOULD be redirected to http://localhost/path/to/your/site and you SHOULD see a message like:
        Incorrect access detected, this server may be accessed only through "http://localhost/path/to/your/site" address, sorry.
        Please notify server administrator.
      8. Run php admin/tool/behat/cli/util.php --enable
      9. It SHOULD finish as expected providing a command to run
      10. Copy & paste the command
      11. Wait until the first scenario finishes (or wait 1 minute instead)
      12. It SHOULD pass
      13. You can stop it know with Cntl + C

      Test 2, switch completely overwrites behat_wwwroot

      1. Set this in your config.php, ensure you don't have the same vars defined below/above in your config file:
        $CFG->wwwroot = 'http://localhost/path/to/your/site';
        $CFG->behat_switchcompletely = true;
        $CFG->behat_wwwroot = http://YOURIP/path/to/your/site
      2. Go to http://localhost/path/to/your/site
      3. You SHOULD see your test site and the page SHOULD be styled as usual
      4. Follow Log in link
      5. You SHOULD be redirected to the login page and the page SHOULD be styled as usual
      6. Go to http://YOURIP/path/to/your/site
      7. You SHOULD be redirected to http://localhost/path/to/your/site and you SHOULD see a message like:
        Incorrect access detected, this server may be accessed only through "http://localhost/path/to/your/site" address, sorry.
        Please notify server administrator.
      8. Run php admin/tool/behat/cli/util.php --enable
      9. It SHOULD finish as expected providing a command to run
      10. Copy & paste the command
      11. Wait until the first scenario finishes (or wait 1 minute instead)
      12. It SHOULD pass
      13. You can stop it know with Cntl + C

      Test 3, behat_wwwroot works as expected

      1. Set this in your config.php, ensure you don't have the same vars defined below/above in your config file:
        $CFG->wwwroot = 'http://localhost/path/to/your/site';
        //$CFG->behat_switchcompletely = true;
        $CFG->behat_wwwroot = http://YOURIP/path/to/your/site
      2. Go to http://localhost/path/to/your/site
      3. You SHOULD see your production site
      4. Go to http://YOURIP/path/to/your/site
      5. You SHOULD see your test site and the page SHOULD be styled as usual
      6. Follow Log in link
      7. You SHOULD be redirected to the login page and the page SHOULD be styled as usual
      8. Go to http://yourmachine.host.name/path/to/your/site
      9. You SHOULD be redirected to http://localhost/path/to/your/site and you SHOULD see a message like:
        Incorrect access detected, this server may be accessed only through "http://localhost/path/to/your/site" address, sorry.
        Please notify server administrator.
      10. Run php admin/tool/behat/cli/util.php --enable
      11. It SHOULD finish as expected providing a command to run
      12. Copy & paste the command
      13. Wait until the first scenario finishes (or wait 1 minute instead)
      14. It SHOULD pass
      15. You can stop it know with Cntl + C

      Test 4, the built-in server defaults to localhost:8000 if no behat_wwwroot is set

      1. Set this in your config.php, ensure you don't have the same vars defined below/above in your config file:
        $CFG->wwwroot = 'http://localhost/path/to/your/site';
        //$CFG->behat_switchcompletely = true;
        //$CFG->behat_wwwroot = http://YOURIP/path/to/your/site
      2. Start your PHP built-in server to the default URL
        cd /path/to/your/dirroot/
        php -S localhost:8000
      3. Go to http://localhost/path/to/your/site
      4. You SHOULD see your production site
      5. Go to http://localhost:8000
      6. You SHOULD see your test site and the page SHOULD be styled as usual
      7. Follow Log in link
      8. You SHOULD be redirected to the login page and the page SHOULD be styled as usual
      9. Run php admin/tool/behat/cli/util.php --enable
      10. It SHOULD finish as expected providing a command to run
      11. Copy & paste the command
      12. Wait until the first scenario finishes (or wait 1 minute instead)
      13. It SHOULD pass
      14. You can stop it know with Cntl + C

      Test 5, the built-in server can use a different URL than localhost:8000

      1. Set this in your config.php, ensure you don't have the same vars defined below/above in your config file:
        $CFG->wwwroot = 'http://localhost/path/to/your/site';
        //$CFG->behat_switchcompletely = true;
        $CFG->behat_wwwroot = http://localhost:8005
      2. Start your PHP built-in server to the default URL
        cd /path/to/your/dirroot/
        php -S localhost:8005
      3. Go to http://localhost/path/to/your/site
      4. You SHOULD see your production site
      5. Go to http://localhost:8005
      6. You SHOULD see your test site and the page SHOULD be styled as usual
      7. Follow Log in link
      8. You SHOULD be redirected to the login page and the page SHOULD be styled as usual
      9. Run php admin/tool/behat/cli/util.php --enable
      10. It SHOULD finish as expected providing a command to run
      11. Copy & paste the command
      12. Wait until the first scenario finishes (or wait 1 minute instead)
      13. It SHOULD pass
      14. You can stop it know with Cntl + C

      Test 6, using the built-in server, behat_wwwroot wins over it's default URL

      1. Set this in your config.php, ensure you don't have the same vars defined below/above in your config file:
        $CFG->wwwroot = 'http://localhost/path/to/your/site';
        //$CFG->behat_switchcompletely = true;
        $CFG->behat_wwwroot = http://YOURIP/path/to/your/site
      2. Start your PHP built-in server to the default URL
        cd /path/to/your/dirroot/
        php -S localhost:8000
      3. Go to http://localhost/path/to/your/site
      4. You SHOULD see your production site
      5. Go to http://localhost:8000
      6. You SHOULD be redirected to http://localhost/path/to/your/site and you SHOULD see a message like:
        Incorrect access detected, this server may be accessed only through "http://localhost/path/to/your/site" address, sorry.
        Please notify server administrator.
      7. Go to http://YOURIP/path/to/your/site
      8. You SHOULD see your test site and the page SHOULD be styled as usual
      9. Follow Log in link
      10. You SHOULD be redirected to the login page and the page SHOULD be styled as usual
      11. Go to http://yourmachine.host.name/path/to/your/site
      12. You SHOULD be redirected to http://localhost/path/to/your/site and you SHOULD see a message like:
        Incorrect access detected, this server may be accessed only through "http://localhost/path/to/your/site" address, sorry.
        Please notify server administrator.
      13. Run php admin/tool/behat/cli/util.php --enable
      14. It SHOULD finish as expected providing a command to run
      15. Copy & paste the command
      16. Wait until the first scenario finishes (or wait 1 minute instead)
      17. It SHOULD pass
      18. You can stop it know with Cntl + C
      Show
      We should test that all that combinations are working Note that as "production site" I mean the normal environment, and as "test site" the one titled as "Acceptance test site". Test 0, install behat Install behat or util.php --drop & util.php --install & util.php --enable if it was already installed It SHOULD finish as expected Test 1, switch completely works as expected Set this in your config.php, ensure you don't have the same vars defined below/above in your config file: $CFG->wwwroot = 'http://localhost/path/to/your/site'; $CFG->behat_switchcompletely = true; //$CFG->behat_wwwroot = http://YOURIP/path/to/your/site Go to http://localhost/path/to/your/site You SHOULD see your test site and the page SHOULD be styled as usual Follow Log in link You SHOULD be redirected to the login page and the page SHOULD be styled as usual Go to http://YOURIP/path/to/your/site You SHOULD be redirected to http://localhost/path/to/your/site and you SHOULD see a message like: Incorrect access detected, this server may be accessed only through "http://localhost/path/to/your/site" address, sorry. Please notify server administrator. Run php admin/tool/behat/cli/util.php --enable It SHOULD finish as expected providing a command to run Copy & paste the command Wait until the first scenario finishes (or wait 1 minute instead) It SHOULD pass You can stop it know with Cntl + C Test 2, switch completely overwrites behat_wwwroot Set this in your config.php, ensure you don't have the same vars defined below/above in your config file: $CFG->wwwroot = 'http://localhost/path/to/your/site'; $CFG->behat_switchcompletely = true; $CFG->behat_wwwroot = http://YOURIP/path/to/your/site Go to http://localhost/path/to/your/site You SHOULD see your test site and the page SHOULD be styled as usual Follow Log in link You SHOULD be redirected to the login page and the page SHOULD be styled as usual Go to http://YOURIP/path/to/your/site You SHOULD be redirected to http://localhost/path/to/your/site and you SHOULD see a message like: Incorrect access detected, this server may be accessed only through "http://localhost/path/to/your/site" address, sorry. Please notify server administrator. Run php admin/tool/behat/cli/util.php --enable It SHOULD finish as expected providing a command to run Copy & paste the command Wait until the first scenario finishes (or wait 1 minute instead) It SHOULD pass You can stop it know with Cntl + C Test 3, behat_wwwroot works as expected Set this in your config.php, ensure you don't have the same vars defined below/above in your config file: $CFG->wwwroot = 'http://localhost/path/to/your/site'; //$CFG->behat_switchcompletely = true; $CFG->behat_wwwroot = http://YOURIP/path/to/your/site Go to http://localhost/path/to/your/site You SHOULD see your production site Go to http://YOURIP/path/to/your/site You SHOULD see your test site and the page SHOULD be styled as usual Follow Log in link You SHOULD be redirected to the login page and the page SHOULD be styled as usual Go to http://yourmachine.host.name/path/to/your/site You SHOULD be redirected to http://localhost/path/to/your/site and you SHOULD see a message like: Incorrect access detected, this server may be accessed only through "http://localhost/path/to/your/site" address, sorry. Please notify server administrator. Run php admin/tool/behat/cli/util.php --enable It SHOULD finish as expected providing a command to run Copy & paste the command Wait until the first scenario finishes (or wait 1 minute instead) It SHOULD pass You can stop it know with Cntl + C Test 4, the built-in server defaults to localhost:8000 if no behat_wwwroot is set Set this in your config.php, ensure you don't have the same vars defined below/above in your config file: $CFG->wwwroot = 'http://localhost/path/to/your/site'; //$CFG->behat_switchcompletely = true; //$CFG->behat_wwwroot = http://YOURIP/path/to/your/site Start your PHP built-in server to the default URL cd /path/to/your/dirroot/ php -S localhost:8000 Go to http://localhost/path/to/your/site You SHOULD see your production site Go to http://localhost:8000 You SHOULD see your test site and the page SHOULD be styled as usual Follow Log in link You SHOULD be redirected to the login page and the page SHOULD be styled as usual Run php admin/tool/behat/cli/util.php --enable It SHOULD finish as expected providing a command to run Copy & paste the command Wait until the first scenario finishes (or wait 1 minute instead) It SHOULD pass You can stop it know with Cntl + C Test 5, the built-in server can use a different URL than localhost:8000 Set this in your config.php, ensure you don't have the same vars defined below/above in your config file: $CFG->wwwroot = 'http://localhost/path/to/your/site'; //$CFG->behat_switchcompletely = true; $CFG->behat_wwwroot = http://localhost:8005 Start your PHP built-in server to the default URL cd /path/to/your/dirroot/ php -S localhost:8005 Go to http://localhost/path/to/your/site You SHOULD see your production site Go to http://localhost:8005 You SHOULD see your test site and the page SHOULD be styled as usual Follow Log in link You SHOULD be redirected to the login page and the page SHOULD be styled as usual Run php admin/tool/behat/cli/util.php --enable It SHOULD finish as expected providing a command to run Copy & paste the command Wait until the first scenario finishes (or wait 1 minute instead) It SHOULD pass You can stop it know with Cntl + C Test 6, using the built-in server, behat_wwwroot wins over it's default URL Set this in your config.php, ensure you don't have the same vars defined below/above in your config file: $CFG->wwwroot = 'http://localhost/path/to/your/site'; //$CFG->behat_switchcompletely = true; $CFG->behat_wwwroot = http://YOURIP/path/to/your/site Start your PHP built-in server to the default URL cd /path/to/your/dirroot/ php -S localhost:8000 Go to http://localhost/path/to/your/site You SHOULD see your production site Go to http://localhost:8000 You SHOULD be redirected to http://localhost/path/to/your/site and you SHOULD see a message like: Incorrect access detected, this server may be accessed only through "http://localhost/path/to/your/site" address, sorry. Please notify server administrator. Go to http://YOURIP/path/to/your/site You SHOULD see your test site and the page SHOULD be styled as usual Follow Log in link You SHOULD be redirected to the login page and the page SHOULD be styled as usual Go to http://yourmachine.host.name/path/to/your/site You SHOULD be redirected to http://localhost/path/to/your/site and you SHOULD see a message like: Incorrect access detected, this server may be accessed only through "http://localhost/path/to/your/site" address, sorry. Please notify server administrator. Run php admin/tool/behat/cli/util.php --enable It SHOULD finish as expected providing a command to run Copy & paste the command Wait until the first scenario finishes (or wait 1 minute instead) It SHOULD pass You can stop it know with Cntl + C
    • Affected Branches:
      MOODLE_25_STABLE
    • Fixed Branches:
      MOODLE_25_STABLE, MOODLE_26_STABLE
    • Pull Master Branch:
      MDL-41592_master

      Description

      The current methods for running Behat tests with PHP 5.3 and 5.4+ are both a bit clunky. Sure, they work for all users but don't really handle more advanced use cases.

      This patch allows you to define an alternate wwwroot for Behat. For example, creating a second Hostname/ServerAlias pointing to the same document root. No more need to boot up the in-built PHP webserver or toggle switchcompletely every time you want to run tests.

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:
                  Fix Release Date:
                  13/Jan/14