Moodle
  1. Moodle
  2. MDL-42942

Add some environmental information at the beginning of every behat run

    Details

    • Testing Instructions:
      Hide

      Test 1

      1. Run unit tests, you can stop them after a while, we just need to check that the site info is correctly printed.

      Test 2

      1. Update your behat site using admin/tool/behat/cli/init.php, it will download the latest moodlehq/behat-extension version
      2. Run behat with the default options (only --config)
      3. You SHOULD see something like this (you can stop the command, there is no need to run the suite)
        Moodle 2.Ndev (Build: 2013NNNN), pgsql, IAMAHASH
        Server OS "Linux", Browser: "firefox"
      4. Check that all the info is correct:
        • Moodle version
        • Database engine
        • Last commit hash
        • Your operative system
        • The browser you run (should be firefox as it is the default one)
      5. Run behat using --format pretty option
      6. The tests SHOULD begin running and you SHOULD NOT see the same header with the data above
      Show
      Test 1 Run unit tests, you can stop them after a while, we just need to check that the site info is correctly printed. Test 2 Update your behat site using admin/tool/behat/cli/init.php, it will download the latest moodlehq/behat-extension version Run behat with the default options (only --config) You SHOULD see something like this (you can stop the command, there is no need to run the suite) Moodle 2.Ndev (Build: 2013NNNN), pgsql, IAMAHASH Server OS "Linux", Browser: "firefox" Check that all the info is correct: Moodle version Database engine Last commit hash Your operative system The browser you run (should be firefox as it is the default one) Run behat using --format pretty option The tests SHOULD begin running and you SHOULD NOT see the same header with the data above
    • Affected Branches:
      MOODLE_25_STABLE, MOODLE_26_STABLE, MOODLE_27_STABLE
    • Fixed Branches:
      MOODLE_25_STABLE, MOODLE_26_STABLE
    • Pull from Repository:
    • Pull 2.6 Branch:
    • Pull Master Branch:
      MDL-42942_master

      Description

      Very similar to the way we do it in unit tests:

      Moodle 2.7dev (Build: 20131117), pgsql, b6813d8f6aff2f24782c1679899c561dfa234576
      PHPUnit 3.7.28 by Sebastian Bergmann.
       
      Configuration read from /Users/Shared/Jenkins/Home/git_repositories/master/phpunit.xml
       
      .............................................................   61 / 2358 (  2%)
      .............................................................  122 / 2358 (  5%)
      ...
      ...
      

      it would be interesting to have this information printed on each execution of behat tests (for easier identifying the run specifics):

      • Moodle release
      • DB type
      • current commit (if the site is git-based)
      • webdriver and OS where it's being executed.

      Ciao

        Gliffy Diagrams

          Issue Links

            Activity

            Hide
            David Monllaó added a comment -

            We need to extend the ProgressFormatter and probably the logger or other's behat base classes.

            Show
            David Monllaó added a comment - We need to extend the ProgressFormatter and probably the logger or other's behat base classes.
            Hide
            David Monllaó added a comment -

            Most of the changes are in moodlehq/behat-extension side (https://github.com/moodlehq/moodle-behat-extension/compare/v1.27.0...master) there is a new commit in each major branch but will not be pulled by composer until we create the tag.

            The changes in moodle-behat-extension and this MDL have dependencies between them, so one without the other will block behat from running. I would create the new tags for https://github.com/moodlehq/moodle-behat-extension once it gets integrated, the only affected users will be the ones that installs behat for the first time during the interval between integration and Friday release, the other option would be to delay it until it passes testing (changing the testing instructions to force the tester to pull the branch rather than the last tag returned by composer) The last option would be to release the tags after the release, we would also need to update the testing instructions but users updating behat would also be affected.

            git clone git@github.com:moodlehq/moodle-behat-extension
            cd moodle-behat-extension
            git checkout master
            git tag -a v1.27.1 -m "v1.27.1"
            git checkout MOODLE_26_STABLE
            git tag -a v1.26.3 -m "v1.26.3"
            git checkout MOODLE_25_STABLE
            git tag -a v1.25.7 -m "v1.25.7"
            

            Show
            David Monllaó added a comment - Most of the changes are in moodlehq/behat-extension side ( https://github.com/moodlehq/moodle-behat-extension/compare/v1.27.0...master ) there is a new commit in each major branch but will not be pulled by composer until we create the tag. The changes in moodle-behat-extension and this MDL have dependencies between them, so one without the other will block behat from running. I would create the new tags for https://github.com/moodlehq/moodle-behat-extension once it gets integrated, the only affected users will be the ones that installs behat for the first time during the interval between integration and Friday release, the other option would be to delay it until it passes testing (changing the testing instructions to force the tester to pull the branch rather than the last tag returned by composer) The last option would be to release the tags after the release, we would also need to update the testing instructions but users updating behat would also be affected. git clone git@github.com:moodlehq/moodle-behat-extension cd moodle-behat-extension git checkout master git tag -a v1.27.1 -m "v1.27.1" git checkout MOODLE_26_STABLE git tag -a v1.26.3 -m "v1.26.3" git checkout MOODLE_25_STABLE git tag -a v1.25.7 -m "v1.25.7"
            Hide
            David Monllaó added a comment -

            Please include https://github.com/moodlehq/moodle-behat-extension/commit/8ddbf5fc800edac56887eae169c940d4f39def61 in the review as far as possible. It not a big change and IMO is the best solution to avoid extending many components of behat.

            I've chosen calling the moodle codebase methods from the moodle behat extension over extending many behat components as future issues when upgrading behat dependencies would be much harder to detect extending the framework.

            Show
            David Monllaó added a comment - Please include https://github.com/moodlehq/moodle-behat-extension/commit/8ddbf5fc800edac56887eae169c940d4f39def61 in the review as far as possible. It not a big change and IMO is the best solution to avoid extending many components of behat. I've chosen calling the moodle codebase methods from the moodle behat extension over extending many behat components as future issues when upgrading behat dependencies would be much harder to detect extending the framework.
            Hide
            Petr Skoda added a comment -

            I like this, +1 for integration, I have fixed the diff url

            and please add test instructions for phpunit execution too because the code flow there is affected by this change

            Show
            Petr Skoda added a comment - I like this, +1 for integration, I have fixed the diff url and please add test instructions for phpunit execution too because the code flow there is affected by this change
            Hide
            David Monllaó added a comment -

            Thanks Petr, done.

            Note for integrator: Composer downloads and uses the last tagged version (http://getcomposer.org/doc/01-basic-usage.md#package-versions), to test the patch you will need to pull the latest moodlehq/moodle-behat-extension code; the latest master, MOODLE_26_STABLE, MOODLE_25_STABLE hashes differs from the last tagged ones (the ones used by composer) so to get the latest version you need to:

            1. Usual composer install (probably already have a populated vendor/, you can skip it in this case)
            1. Update vendor/moodlehq/behat-extension to the latest (master|MOODLE_26_STABLE|MOODLE_25_STABLE) version
              cd vendor/moodlehq
              rm -rf behat-extension
              git clone git://github.com/moodlehq/moodle-behat-extension.git behat-extension
              cd behat-extension
              git checkout NAME_OF_THE_BRANCH_YOU_ARE_TESTING
            Show
            David Monllaó added a comment - Thanks Petr, done. Note for integrator: Composer downloads and uses the last tagged version ( http://getcomposer.org/doc/01-basic-usage.md#package-versions ), to test the patch you will need to pull the latest moodlehq/moodle-behat-extension code; the latest master, MOODLE_26_STABLE, MOODLE_25_STABLE hashes differs from the last tagged ones (the ones used by composer) so to get the latest version you need to: Usual composer install (probably already have a populated vendor/, you can skip it in this case) Update vendor/moodlehq/behat-extension to the latest (master|MOODLE_26_STABLE|MOODLE_25_STABLE) version cd vendor/moodlehq rm -rf behat-extension git clone git://github.com/moodlehq/moodle-behat-extension.git behat-extension cd behat-extension git checkout NAME_OF_THE_BRANCH_YOU_ARE_TESTING
            Hide
            Dan Poltawski added a comment -

            Hi David,

            As commented in the office - we need to make sure that any updates to the moodlehq/moodle-behat-extension DO NOT break existing installs. If a university wants to test their 2.5.3 install it must always be constantly working and be able to be tested against.

            This issue worries me, because it would break that from working. Anyway, as discussed, you are going to address that versioning.

            Show
            Dan Poltawski added a comment - Hi David, As commented in the office - we need to make sure that any updates to the moodlehq/moodle-behat-extension DO NOT break existing installs. If a university wants to test their 2.5.3 install it must always be constantly working and be able to be tested against. This issue worries me, because it would break that from working. Anyway, as discussed, you are going to address that versioning.
            Hide
            David Monllaó added a comment -

            Thanks Dan, I've created https://tracker.moodle.org/browse/MDL-43050 and send it to PR, marking it as blocker for this one

            Show
            David Monllaó added a comment - Thanks Dan, I've created https://tracker.moodle.org/browse/MDL-43050 and send it to PR, marking it as blocker for this one
            Hide
            David Monllaó added a comment -

            Pull branches updated and 26 version on top of MDL-43050, which sets the 1.26 version to 1.26.2 (the current one) waiting for the issue to be integrated to create 1.27.1, 1.26.3, and 1.25.7; we could create this last one right now as in 2.5 the version is already hardcoded but I will wait to do it all at the same time.

            Show
            David Monllaó added a comment - Pull branches updated and 26 version on top of MDL-43050 , which sets the 1.26 version to 1.26.2 (the current one) waiting for the issue to be integrated to create 1.27.1, 1.26.3, and 1.25.7; we could create this last one right now as in 2.5 the version is already hardcoded but I will wait to do it all at the same time.
            Hide
            Dan Poltawski added a comment -

            Just to summarise a discussion David, Damyon and I had in the office regarding the behat extension..

            1/ We should not make backwards-incompatible changes to the behat extension on the stable branches.
            2/ If we have to make backwards-incompatible changes, we should document them
            3/ If we do need to make backwards-incompatible changes, then that is a suggestion that the issue should not be back ported
            4/ Fixing the versions in composer file (MDL-43050) does mean this is not an issue, but there may be cases where these rules help us in the future. (And of course, it'll help with 2.6.0, which was released with a wild carded version)

            Show
            Dan Poltawski added a comment - Just to summarise a discussion David, Damyon and I had in the office regarding the behat extension.. 1/ We should not make backwards-incompatible changes to the behat extension on the stable branches. 2/ If we have to make backwards-incompatible changes, we should document them 3/ If we do need to make backwards-incompatible changes, then that is a suggestion that the issue should not be back ported 4/ Fixing the versions in composer file ( MDL-43050 ) does mean this is not an issue, but there may be cases where these rules help us in the future. (And of course, it'll help with 2.6.0, which was released with a wild carded version)
            Hide
            David Monllaó added a comment -

            Hi Dan,

            When I have seen the code I realized that I already did it backwards compatible (I have more instinct than memory)

            I've created and updated the tags, so now we have: v1.27.0, v1.26.3, v1.25.7 and they can be pulled by composer without breaking compatibility, I've tried it using 2.6 branch. 1.27.0 will not be used until MDL-42738 gets integrated (BTW, I'm updating again MDL-42738 on top of this issue's master pull branch)

            Show
            David Monllaó added a comment - Hi Dan, When I have seen the code I realized that I already did it backwards compatible (I have more instinct than memory) I've created and updated the tags, so now we have: v1.27.0, v1.26.3, v1.25.7 and they can be pulled by composer without breaking compatibility, I've tried it using 2.6 branch. 1.27.0 will not be used until MDL-42738 gets integrated (BTW, I'm updating again MDL-42738 on top of this issue's master pull branch)
            Hide
            Dan Poltawski added a comment -

            Integrated to master, 26 and 25 - thanks David

            Show
            Dan Poltawski added a comment - Integrated to master, 26 and 25 - thanks David
            Hide
            Dan Poltawski added a comment -

            All looks good, tested with multiple versions, browsers and databases.

            Show
            Dan Poltawski added a comment - All looks good, tested with multiple versions, browsers and databases.
            Hide
            Dan Poltawski added a comment -

            Congratulations, this change has now made its way upstream. Thanks for your contribution!

            “ Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live. ” - Rick Osborne

            Show
            Dan Poltawski added a comment - Congratulations, this change has now made its way upstream. Thanks for your contribution! “ Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live. ” - Rick Osborne

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: