Moodle
  1. Moodle
  2. MDL-45699

Allow acceptance tests to be executed against custom cache stores.

    Details

    • Testing Instructions:
      Hide

      Hi.
      It's my last week here, on Friday I finish.
      Unfortunately you the tester are going to hate me on my last week....

      Enjoy

      You have to do this locally sorry, CI servers can't do this for you as its imperative you check locally.

      Setup:

      1. Make sure you have MongoDB and its PHP extension installed.
      2. Make sure you have Memcached and its PHP extension installed.
      3. Ensure you don't have either of these set in your config.php
        • TEST_CACHE_USING_ALT_CACHE_CONFIG_PATH
        • TEST_CACHE_USING_APPLICATION_STORE
      4. Add the following to your config.php
        • $CFG->behat_extraallowedsettings = array('perfdebug');
        • $CFG->perfdebug = 15;
        • $CFG->altcacheconfigpath = '/tmp/testing-45699.cache';

      Test the default.

      1. In your browser browse to your behat site
      2. Scroll down and verify you see "cachestore_file" being used caches (in the perf debug info)
      3. Initialise phpunit
      4. Initialise behat
      5. Run full phpunit
      6. Run behat --name "Test course pagination"
      7. Make sure both complete successfully.

      Test altcacheconfig

      1. Add define('TEST_CACHE_USING_ALT_CACHE_CONFIG_PATH', true); to your config.php
      2. Browse to your site and log in as admin.
      3. Browse to Site administration > Plugins > Caching > Cache configuration
      4. Set up a MongoDB cache store instance using the defaults
      5. Map your MongoDB cache store as the default application store
      6. Verify that you see MongoDB being used as the cache store for all application caches (in the perfdebug)
      7. Run full phpunit
      8. Run behat --name "Test course pagination"
      9. Make sure both complete successfully.

      Test specific cache testing

      1. Remove the TEST_CACHE_USING_ALT_CACHE_CONFIG_PATH from your config.php
      2. Add the following to your config.php
        • define('TEST_CACHESTORE_MEMCACHED_TESTSERVERS', '127.0.0.1:11211');
        • define('TEST_CACHE_USING_APPLICATION_STORE', 'memcached');
      3. Delete moodledata/muc/config.php
      4. Browse to your site.
      5. Check if the perfconf that you see memcached being used for application caches.
      6. Drop and initiailise phpunit
      7. Drop and initiailise behat
      8. Run full phpunit
      9. Run behat --name "Test course pagination"
      10. Make sure both complete successfully.

      Verify normal use (clean, default site, without any phpunit/behat CFG settings).

      1. Verify Moodle 2.8 and master install ok (CLI and web).
      2. Verify Moodle 2.8 and master upgrade ok (from 2.7, CLI and web).
      3. Visit places using caches and verify there are no problems (groups, admin settings...).
      4. Verify cache configuration works and can add instances, modify them...exploratory testing.
      Show
      Hi. It's my last week here, on Friday I finish. Unfortunately you the tester are going to hate me on my last week.... Enjoy You have to do this locally sorry, CI servers can't do this for you as its imperative you check locally. Setup: Make sure you have MongoDB and its PHP extension installed. Make sure you have Memcached and its PHP extension installed. Ensure you don't have either of these set in your config.php TEST_CACHE_USING_ALT_CACHE_CONFIG_PATH TEST_CACHE_USING_APPLICATION_STORE Add the following to your config.php $CFG->behat_extraallowedsettings = array('perfdebug'); $CFG->perfdebug = 15; $CFG->altcacheconfigpath = '/tmp/testing-45699.cache'; Test the default. In your browser browse to your behat site Scroll down and verify you see "cachestore_file" being used caches (in the perf debug info) Initialise phpunit Initialise behat Run full phpunit Run behat --name "Test course pagination" Make sure both complete successfully. Test altcacheconfig Add define('TEST_CACHE_USING_ALT_CACHE_CONFIG_PATH', true); to your config.php Browse to your site and log in as admin. Browse to Site administration > Plugins > Caching > Cache configuration Set up a MongoDB cache store instance using the defaults Map your MongoDB cache store as the default application store Verify that you see MongoDB being used as the cache store for all application caches (in the perfdebug) Run full phpunit Run behat --name "Test course pagination" Make sure both complete successfully. Test specific cache testing Remove the TEST_CACHE_USING_ALT_CACHE_CONFIG_PATH from your config.php Add the following to your config.php define('TEST_CACHESTORE_MEMCACHED_TESTSERVERS', '127.0.0.1:11211'); define('TEST_CACHE_USING_APPLICATION_STORE', 'memcached'); Delete moodledata/muc/config.php Browse to your site. Check if the perfconf that you see memcached being used for application caches. Drop and initiailise phpunit Drop and initiailise behat Run full phpunit Run behat --name "Test course pagination" Make sure both complete successfully. Verify normal use (clean, default site, without any phpunit/behat CFG settings). Verify Moodle 2.8 and master install ok (CLI and web). Verify Moodle 2.8 and master upgrade ok (from 2.7, CLI and web). Visit places using caches and verify there are no problems (groups, admin settings...). Verify cache configuration works and can add instances, modify them...exploratory testing.
    • Affected Branches:
      MOODLE_27_STABLE, MOODLE_28_STABLE, MOODLE_29_STABLE
    • Fixed Branches:
      MOODLE_28_STABLE
    • Pull 2.8 Branch:
      45699-28
    • Pull Master Branch:
      45699-29
    • Pull Master Diff URL:

      Description

      This is a follow on from MDL-45513 where we enabled phpunit tests to be run against custom cache stores using either $CFG->altcacheconfigpath OR special defines to be used (see cache/README.md)

      It would be nice to be able to run behat tests against specific cache stores and cache configurations.
      This would both facilitate faster runs (hopefully) as well as allowing sites a means by which to test cache configuration and perhaps even its performance in a limited sense.

        Gliffy Diagrams

          Issue Links

            Activity

            Hide
            Sam Hemelryk added a comment -

            Copied from MDL-45513:

            I am not sure whether I will port the defines for behat however.
            The cache API loads special infrastructure when phpunit tests are running in order to facilitate the API testing we do.
            At present no such infrastructure is used for behat, its not needed because of behats end to end nature.
            The altcacheconfigpath route works easily as that relies on core infrastructure.
            We can discuss that more on the new issue of course.

            Show
            Sam Hemelryk added a comment - Copied from MDL-45513 : I am not sure whether I will port the defines for behat however. The cache API loads special infrastructure when phpunit tests are running in order to facilitate the API testing we do. At present no such infrastructure is used for behat, its not needed because of behats end to end nature. The altcacheconfigpath route works easily as that relies on core infrastructure. We can discuss that more on the new issue of course.
            Hide
            Sam Hemelryk added a comment -

            Oops lost this issue for a while, found while debugging MDL-37072 (this issue can be used to test that).

            Show
            Sam Hemelryk added a comment - Oops lost this issue for a while, found while debugging MDL-37072 (this issue can be used to test that).
            Hide
            CiBoT added a comment -

            Code verified against automated checks.

            Checked MDL-45699 using repository: git://github.com/samhemelryk/moodle.git

            More information about this report

            Show
            CiBoT added a comment - Code verified against automated checks. Checked MDL-45699 using repository: git://github.com/samhemelryk/moodle.git MOODLE_27_STABLE (0 errors / 0 warnings) [branch: 45699-27 | CI Job ] MOODLE_28_STABLE (0 errors / 0 warnings) [branch: 45699-28 | CI Job ] master (0 errors / 0 warnings) [branch: 45699-29 | CI Job ] More information about this report
            Hide
            Sam Hemelryk added a comment -

            On second thoughts, this is a simple patch and identical to what Rajesh Taneja would have done last week to run acceptance tests against MongoDB.
            I'm going to move this forward to integration review, for confidence on this I am sure Raj will be happy to review it.

            Show
            Sam Hemelryk added a comment - On second thoughts, this is a simple patch and identical to what Rajesh Taneja would have done last week to run acceptance tests against MongoDB. I'm going to move this forward to integration review, for confidence on this I am sure Raj will be happy to review it.
            Hide
            Rajesh Taneja added a comment -

            Thanks Sam,

            I did set following to make use of custom cache

            $CFG->altcacheconfigpath = '/home/rajesh/faildump';
            $CFG->behat_extraallowedsettings = array('altcacheconfigpath');
            

            Patch will help not to use the former CFG, but that might not be expected.
            As this is same as production cache store, it might not be acceptable.

            Probably another behat specific config might be a nice way.

            Show
            Rajesh Taneja added a comment - Thanks Sam, I did set following to make use of custom cache $CFG->altcacheconfigpath = '/home/rajesh/faildump'; $CFG->behat_extraallowedsettings = array('altcacheconfigpath'); Patch will help not to use the former CFG, but that might not be expected. As this is same as production cache store, it might not be acceptable. Probably another behat specific config might be a nice way.
            Hide
            CiBoT added a comment -

            Moving this issue to current integration cycle, will be reviewed soon. Thanks for the hard work!

            Show
            CiBoT added a comment - Moving this issue to current integration cycle, will be reviewed soon. Thanks for the hard work!
            Hide
            Eloy Lafuente (stronk7) added a comment - - edited

            So TEST_CACHE_USING_APPLICATION_STORE does apply to acceptance tests or no? I had thought it should, but patch does not show any change about that... so bit lost.

            Edited: Looking to code... it seems that create_default_configuration() supports it, am I correct? Or that only applies to unit tests?

            Show
            Eloy Lafuente (stronk7) added a comment - - edited So TEST_CACHE_USING_APPLICATION_STORE does apply to acceptance tests or no? I had thought it should, but patch does not show any change about that... so bit lost. Edited: Looking to code... it seems that create_default_configuration() supports it, am I correct? Or that only applies to unit tests?
            Hide
            Eloy Lafuente (stronk7) added a comment -

            Just checked it here with the define pointing to memcached instance and my acceptance site did not show such cache configured but the default FS one instead. Uhm...

            I really think this should cover both alternatives, aka $CFG->altcacheconfig and TEST_CACHE_USING_APPLICATION_STORE. To be 100% on par with unit tests. That was the original idea when this followup was created.

            Asking for more integrators opinion about this...

            Show
            Eloy Lafuente (stronk7) added a comment - Just checked it here with the define pointing to memcached instance and my acceptance site did not show such cache configured but the default FS one instead. Uhm... I really think this should cover both alternatives, aka $CFG->altcacheconfig and TEST_CACHE_USING_APPLICATION_STORE. To be 100% on par with unit tests. That was the original idea when this followup was created. Asking for more integrators opinion about this...
            Hide
            Sam Hemelryk added a comment -

            I pondered this a bit last night, acceptance tests are black box, at present we have no interception point for acceptance tests in the cache API.
            Unit tests have a special cache config class that gets used instead of the standard config class while unit tests are running, the logic for those defines is wrapped into that config class.
            I suppose the easiest way to allow those defines to function here would be to utilise the unit test config class during acceptance tests as well.
            Let me see how that functions quickly now.

            Show
            Sam Hemelryk added a comment - I pondered this a bit last night, acceptance tests are black box, at present we have no interception point for acceptance tests in the cache API. Unit tests have a special cache config class that gets used instead of the standard config class while unit tests are running, the logic for those defines is wrapped into that config class. I suppose the easiest way to allow those defines to function here would be to utilise the unit test config class during acceptance tests as well. Let me see how that functions quickly now.
            Hide
            Sam Hemelryk added a comment -

            Ok after discussing with Eloy I've come up with a solution to make caching function the identically for unit tests and acceptance tests.
            The solution is 28 and master only, 27 doesn't have the required defines.

            All yours again Eloy!

            Show
            Sam Hemelryk added a comment - Ok after discussing with Eloy I've come up with a solution to make caching function the identically for unit tests and acceptance tests. The solution is 28 and master only, 27 doesn't have the required defines. All yours again Eloy!
            Hide
            Eloy Lafuente (stronk7) added a comment -

            I'm happy hating you already, LOL. Thanks!

            Show
            Eloy Lafuente (stronk7) added a comment - I'm happy hating you already, LOL. Thanks!
            Hide
            Eloy Lafuente (stronk7) added a comment -

            While I'm using this to test MDL-37072... and things are going well so far... I'm afraid I ran out from time to complete this issue testing. So I'll keep it under integration to complete its testing once we roll this round weeklies.

            Sorry for the delay, but prefer it to have 100% tested/covered before integrating it.

            Ciao

            Show
            Eloy Lafuente (stronk7) added a comment - While I'm using this to test MDL-37072 ... and things are going well so far... I'm afraid I ran out from time to complete this issue testing. So I'll keep it under integration to complete its testing once we roll this round weeklies. Sorry for the delay, but prefer it to have 100% tested/covered before integrating it. Ciao
            Hide
            Eloy Lafuente (stronk7) added a comment -

            I must be missing something (obvious), because no matter I set both the define and the CFG and then configure the site to use a mongodb/memcached application store... once the behat run begins... I get file stores being used all the time.

            Also, just setting mongodb as application store leads here to tons of PHP Warnings about unset() problems (in the static accelerators).

            Becoming late here... will try this again tomorrow.... niao!

            Show
            Eloy Lafuente (stronk7) added a comment - I must be missing something (obvious), because no matter I set both the define and the CFG and then configure the site to use a mongodb/memcached application store... once the behat run begins... I get file stores being used all the time. Also, just setting mongodb as application store leads here to tons of PHP Warnings about unset() problems (in the static accelerators). Becoming late here... will try this again tomorrow.... niao!
            Show
            Rajesh Taneja added a comment - Hello Eloy, Seems you missed picking commits from MDL-37072 . https://tracker.moodle.org/browse/MDL-37072?focusedCommentId=327705&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-327705
            Hide
            David Monllaó added a comment -

            Hi, I've tested the behat part (without running unit tests) and all is working as described, I just need to go to my behat_wwwroot to see the perf info in the footer. Using the setting below in config.php I don't even need to set $CFG->debug or MDL_PERF contants.

            $CFG->behat_extraallowedsettings = array('perfdebug');
            $CFG->perfdebug = 15; 
            $CFG->altcacheconfigpath = '/tmp/testing-45699.cache';
            

            Show
            David Monllaó added a comment - Hi, I've tested the behat part (without running unit tests) and all is working as described, I just need to go to my behat_wwwroot to see the perf info in the footer. Using the setting below in config.php I don't even need to set $CFG->debug or MDL_PERF contants. $CFG->behat_extraallowedsettings = array('perfdebug'); $CFG->perfdebug = 15; $CFG->altcacheconfigpath = '/tmp/testing-45699.cache';
            Hide
            Eloy Lafuente (stronk7) added a comment -

            Hi Rajesh Taneja, why are those commits related to this? You mean the PHP warnings side of my comment?

            Ho David Monllaó, are you 100% sure that the behat execution was showing "memcached" or "mongodb" in the footer information? I double-ask because, yes. I was able to see them when visiting the acceptance site manually... but later executions were back to show "file" as application store. And more yet, I was unable to find the $CFG->altcacheconfigpath file.

            Also... did it work for you without setting the TEST_CACHE_USING_ALT_CACHE_CONFIG_PATH define?

            Show
            Eloy Lafuente (stronk7) added a comment - Hi Rajesh Taneja , why are those commits related to this? You mean the PHP warnings side of my comment? Ho David Monllaó , are you 100% sure that the behat execution was showing "memcached" or "mongodb" in the footer information? I double-ask because, yes. I was able to see them when visiting the acceptance site manually... but later executions were back to show "file" as application store. And more yet, I was unable to find the $CFG->altcacheconfigpath file. Also... did it work for you without setting the TEST_CACHE_USING_ALT_CACHE_CONFIG_PATH define?
            Hide
            David Monllaó added a comment -

            Hi Eloy,

            Yes (I will recheck it right now) during the executing of the scenario it was showing cache_mongodb, but now I'm thinking that I didn't check the second scenario (after the database & dataroot reset) it may be resetting the tmp cache file.

            /tmp/testing....cache file was created containing a $CFG->dataroot/muc/config.php format.

            Show
            David Monllaó added a comment - Hi Eloy, Yes (I will recheck it right now) during the executing of the scenario it was showing cache_mongodb, but now I'm thinking that I didn't check the second scenario (after the database & dataroot reset) it may be resetting the tmp cache file. /tmp/testing....cache file was created containing a $CFG->dataroot/muc/config.php format.
            Hide
            David Monllaó added a comment -

            Just rechecked and confirmed what I commented Yesterday, I can see cachestore_memcached in the footer even after resetting and running another scenario; attaching my config below, maybe it is conflicting with one of your eloy-config.php settings?

            $CFG->behat_extraallowedsettings = array('perfdebug');
            $CFG->perfdebug = 15; 
            $CFG->altcacheconfigpath = '/tmp/testsdasing-45699.cache';
             
            define('TEST_CACHESTORE_MEMCACHED_TESTSERVERS', '127.0.0.1:11211');
            define('TEST_CACHE_USING_APPLICATION_STORE', 'memcached');
            

            Show
            David Monllaó added a comment - Just rechecked and confirmed what I commented Yesterday, I can see cachestore_memcached in the footer even after resetting and running another scenario; attaching my config below, maybe it is conflicting with one of your eloy-config.php settings? $CFG->behat_extraallowedsettings = array('perfdebug'); $CFG->perfdebug = 15; $CFG->altcacheconfigpath = '/tmp/testsdasing-45699.cache';   define('TEST_CACHESTORE_MEMCACHED_TESTSERVERS', '127.0.0.1:11211'); define('TEST_CACHE_USING_APPLICATION_STORE', 'memcached');
            Hide
            Eloy Lafuente (stronk7) added a comment -

            Uhm. No. This works for me perfectly:

            define('TEST_CACHESTORE_MEMCACHED_TESTSERVERS', '127.0.0.1:11211');
            define('TEST_CACHE_USING_APPLICATION_STORE', 'memcached');
            

            And in fact that's the way I use it here by default for phpunit since ages ago. For behat, it shows memcached in the footer information both when visiting the acceptance site manually and when running the tests.

            BUT, It's the other combination:

            define('TEST_CACHE_USING_ALT_CACHE_CONFIG_PATH', true);
            $CFG->altcacheconfigpath = '/tmp/testsdasing-45699.cache';
            

            The one giving me headaches here, because I'm able to set it manually in the acceptance site.... but later when I execute the tests... somehow it's reset to the default file store. That's the problem here, that I'm unable to complete the "Test altcacheconfig" section of the tests because the configured stores are reset on behat execution.

            Ciao

            Show
            Eloy Lafuente (stronk7) added a comment - Uhm. No. This works for me perfectly: define('TEST_CACHESTORE_MEMCACHED_TESTSERVERS', '127.0.0.1:11211'); define('TEST_CACHE_USING_APPLICATION_STORE', 'memcached'); And in fact that's the way I use it here by default for phpunit since ages ago. For behat, it shows memcached in the footer information both when visiting the acceptance site manually and when running the tests. BUT, It's the other combination: define('TEST_CACHE_USING_ALT_CACHE_CONFIG_PATH', true); $CFG->altcacheconfigpath = '/tmp/testsdasing-45699.cache'; The one giving me headaches here, because I'm able to set it manually in the acceptance site.... but later when I execute the tests... somehow it's reset to the default file store. That's the problem here, that I'm unable to complete the "Test altcacheconfig" section of the tests because the configured stores are reset on behat execution. Ciao
            Hide
            David Monllaó added a comment -

            It is working for me, all what I've done step by step:

            1. I set the following in config.php:

              $CFG->behat_extraallowedsettings = array('perfdebug');
              $CFG->perfdebug = 15;
              $CFG->altcacheconfigpath = '/tmp/test-eloysete-cache-123.cache';
              define('TEST_CACHE_USING_ALT_CACHE_CONFIG_PATH', true);
              

            2. php admin/cli/purge_caches.php
            3. Go to my wwwroot -> plugins -> caching -> set a new mongodb cache store instance
            4. Set it as application store
            5. After this I can see cachestore_mongodb in the footer (wwwroot)
            6. In behat_wwwroot I see cachestore_file
            7. php admin/tool/behat/cli/util.php (just in case)
            8. In behat_wwwroot I see cachestore_file
            9. I open /tmp/test-eloysete-cache-123.cache
            10. It contains the mongodb cache store I set in wwwroot
            11. I open usual $CFG->dataroot/muc/config.php
            12. It does not contain the mongodb cache store I set in wwwroot
            13. Go to my behat_wwwroot -> plugins -> caching -> set a new mongodb cache store instance
            14. Set it as application store
            15. In footer I see cachestore_mongo
            16. I start a vendor/bin/behat run
            17. The first scenario starts and cachestore_mongodb is still in the footer
            18. Same after reset after first scenario finishes successfully
            Show
            David Monllaó added a comment - It is working for me, all what I've done step by step: I set the following in config.php: $CFG->behat_extraallowedsettings = array('perfdebug'); $CFG->perfdebug = 15; $CFG->altcacheconfigpath = '/tmp/test-eloysete-cache-123.cache'; define('TEST_CACHE_USING_ALT_CACHE_CONFIG_PATH', true); php admin/cli/purge_caches.php Go to my wwwroot -> plugins -> caching -> set a new mongodb cache store instance Set it as application store After this I can see cachestore_mongodb in the footer (wwwroot) In behat_wwwroot I see cachestore_file php admin/tool/behat/cli/util.php (just in case) In behat_wwwroot I see cachestore_file I open /tmp/test-eloysete-cache-123.cache It contains the mongodb cache store I set in wwwroot I open usual $CFG->dataroot/muc/config.php It does not contain the mongodb cache store I set in wwwroot Go to my behat_wwwroot -> plugins -> caching -> set a new mongodb cache store instance Set it as application store In footer I see cachestore_mongo I start a vendor/bin/behat run The first scenario starts and cachestore_mongodb is still in the footer Same after reset after first scenario finishes successfully
            Hide
            Eloy Lafuente (stronk7) added a comment -

            Thanks for the detailed steps. I'm 90% sure that's the way I've proceed here... going to try again... perhaps i used to perform a drop & init in the middle ... not sure really...

            trying...

            Show
            Eloy Lafuente (stronk7) added a comment - Thanks for the detailed steps. I'm 90% sure that's the way I've proceed here... going to try again... perhaps i used to perform a drop & init in the middle ... not sure really... trying...
            Hide
            Eloy Lafuente (stronk7) added a comment -

            Ok, finally found why I was getting runs back to use file store all the time:

            • My testing site DOES NOT have real sites installed. Just the phpunit and behat ones.
            • So it was impossible for me to execute steps 3-5 (that seems to be the ones creating the alternative config file.
            • So I was setting mongodb/memcached directly in the behat site (and I saw them in the footer). But... that was not writing the alternative config file at all, so once the tests started... on 1st reset... it was changed back to file store.

            So, what I've done is:

            1) change the behat site (the only one I've) to use mongodb/memcached.
            2) copy moodledata/muc/config.php to $CFG->altcacheconfigpath
            3) Yay, working. Tests are executed with the mongodb/memcached stores finally!

            Mystery solved... I continue testing all combinations and branches now. Just guessing if we should consider recommending to, always, perform the changes in behat site and then copy (as far as real site may not exist/be a requirement). Or perhaps we should consider the fact of the behat site not writing the alternative file a bug? (maybe it's on purpose).

            Ciao

            Show
            Eloy Lafuente (stronk7) added a comment - Ok, finally found why I was getting runs back to use file store all the time: My testing site DOES NOT have real sites installed. Just the phpunit and behat ones. So it was impossible for me to execute steps 3-5 (that seems to be the ones creating the alternative config file. So I was setting mongodb/memcached directly in the behat site (and I saw them in the footer). But... that was not writing the alternative config file at all, so once the tests started... on 1st reset... it was changed back to file store. So, what I've done is: 1) change the behat site (the only one I've) to use mongodb/memcached. 2) copy moodledata/muc/config.php to $CFG->altcacheconfigpath 3) Yay, working. Tests are executed with the mongodb/memcached stores finally! Mystery solved... I continue testing all combinations and branches now. Just guessing if we should consider recommending to, always, perform the changes in behat site and then copy (as far as real site may not exist/be a requirement). Or perhaps we should consider the fact of the behat site not writing the alternative file a bug? (maybe it's on purpose). Ciao
            Hide
            Eloy Lafuente (stronk7) added a comment -

            Ok, these are the results of testing with these 3 (highly inter-dependent MDL-45699, MDL-37072 and MDL-48568) in use:

            MOODLE_28_STABLE

            Default:
            phpunit: Ok. Time: 17.43 minutes, Memory: 507.25Mb
            behat: Ok. 209 pasos (209 exitosos). 2m33.574s

            Altconfig mongodb:
            phpunit: 1 FAILURE. Time: 14.85 minutes, Memory: 417.25Mb
            behat: Ok. 209 pasos (209 exitosos) 2m28.838s

            Altconfig memcached:
            phpunit: 1 FAILURE. Time: 14.79 minutes, Memory: 486.75Mb
            behat: Ok. 209 pasos (209 exitosos) 2m28.343s

            Specific cache mongodb:
            phpunit: Ok. Time: 14.6 minutes, Memory: 398.00Mb
            behat: Ok. 209 pasos (209 exitosos) 2m27.302s

            Specific cache memcached:
            phpunit: Ok. Time: 13.26 minutes, Memory: 430.75Mb
            behat: Ok. 209 pasos (209 exitosos) 2m28.547s

            master

            Default:
            phpunit: Ok. Time: 17.11 minutes, Memory: 509.25Mb
            behat: Ok. 209 pasos (209 exitosos) 2m29.454s

            Altconfig mongodb:
            phpunit: 1 FAILURE. Time: 14.92 minutes, Memory: 417.25Mb
            behat: Ok. 209 pasos (209 exitosos) 2m28.696s

            Altconfig memcached:
            phpunit: 1 FAILURE. Time: 14.99 minutes, Memory: 487.75Mb
            behat: Ok. 209 pasos (209 exitosos) 2m28.854s

            Specific cache mongodb:
            phpunit: Ok. Time: 15.21 minutes, Memory: 398.25Mb
            behat: Ok. 209 pasos (209 exitosos) 2m28.646s

            Specific cache memcached:
            phpunit: Ok. Time: 13.65 minutes, Memory: 431.25Mb
            behat: Ok. 209 pasos (209 exitosos) 2m27.898s

            The FOUR failures above correspond to the same test that MDL-48568 has fixed for the “specific cache” (named “1st way”t there). So I’m delegating the fix of those 4 failures using “altconfig caches” (named “2nd way” there) to the very same issue. Will repeat the failing tests there. See this comment and following.

            Given that, I consider this 100% integrable and test-passed once MDL-48568 and MDL-37072 achieve the same status. It’s all-together or none.

            Show
            Eloy Lafuente (stronk7) added a comment - Ok, these are the results of testing with these 3 (highly inter-dependent MDL-45699 , MDL-37072 and MDL-48568 ) in use: MOODLE_28_STABLE Default: phpunit: Ok . Time: 17.43 minutes, Memory: 507.25Mb behat: Ok . 209 pasos (209 exitosos). 2m33.574s Altconfig mongodb: phpunit: 1 FAILURE . Time: 14.85 minutes, Memory: 417.25Mb behat: Ok . 209 pasos (209 exitosos) 2m28.838s Altconfig memcached: phpunit: 1 FAILURE . Time: 14.79 minutes, Memory: 486.75Mb behat: Ok . 209 pasos (209 exitosos) 2m28.343s Specific cache mongodb: phpunit: Ok . Time: 14.6 minutes, Memory: 398.00Mb behat: Ok . 209 pasos (209 exitosos) 2m27.302s Specific cache memcached: phpunit: Ok . Time: 13.26 minutes, Memory: 430.75Mb behat: Ok . 209 pasos (209 exitosos) 2m28.547s master Default: phpunit: Ok . Time: 17.11 minutes, Memory: 509.25Mb behat: Ok . 209 pasos (209 exitosos) 2m29.454s Altconfig mongodb: phpunit: 1 FAILURE . Time: 14.92 minutes, Memory: 417.25Mb behat: Ok . 209 pasos (209 exitosos) 2m28.696s Altconfig memcached: phpunit: 1 FAILURE . Time: 14.99 minutes, Memory: 487.75Mb behat: Ok . 209 pasos (209 exitosos) 2m28.854s Specific cache mongodb: phpunit: Ok . Time: 15.21 minutes, Memory: 398.25Mb behat: Ok . 209 pasos (209 exitosos) 2m28.646s Specific cache memcached: phpunit: Ok . Time: 13.65 minutes, Memory: 431.25Mb behat: Ok . 209 pasos (209 exitosos) 2m27.898s The FOUR failures above correspond to the same test that MDL-48568 has fixed for the “specific cache” (named “1st way”t there). So I’m delegating the fix of those 4 failures using “altconfig caches” (named “2nd way” there) to the very same issue. Will repeat the failing tests there. See this comment and following. Given that, I consider this 100% integrable and test-passed once MDL-48568 and MDL-37072 achieve the same status . It’s all-together or none.
            Hide
            Eloy Lafuente (stronk7) added a comment -

            Ok, we have decided to move still problematic MDL-37072 out from the equation, so some tests are being rerun with only MDL-45699 and MDL-48568 integrated, of course keeping mongodb out and just verifying default and memcache(d) stores.

            The branches being used for re-testing are:

            master: https://github.com/stronk7/moodle/compare/master...MDL-48568_and_MDL-45699
            28_STABLE: https://github.com/stronk7/moodle/compare/MOODLE_28_STABLE...MDL-48568_and_MDL-45699_28

            Ciao

            Show
            Eloy Lafuente (stronk7) added a comment - Ok, we have decided to move still problematic MDL-37072 out from the equation, so some tests are being rerun with only MDL-45699 and MDL-48568 integrated, of course keeping mongodb out and just verifying default and memcache(d) stores. The branches being used for re-testing are: master: https://github.com/stronk7/moodle/compare/master...MDL-48568_and_MDL-45699 28_STABLE: https://github.com/stronk7/moodle/compare/MOODLE_28_STABLE...MDL-48568_and_MDL-45699_28 Ciao
            Hide
            Eloy Lafuente (stronk7) added a comment - - edited

            Ok, these have been tested and passed against the branches in previous comment:

            1) default configuration, not using alternative caches:

            • complete phpunit. 28_STABLE (ok) and master (ok).
            • behat (@core_course,@core_group). 28_STABLE (ok) and master (ok).

            2) configuration using memcached via TEST_CACHE_USING_APPLICATION_STORE

            • complete phpunit. 28_STABLE (ok) and master (ok).
            • behat (@core_course,@core_group). 28_STABLE (ok) and master (ok).

            3) configuration using memcache via $CFG->altcacheconfigpath

            • complete phpunit. 28_STABLE (ok) and master (ok).
            • behat (@core_course,@core_group). 28_STABLE (ok) and master (ok).
            Show
            Eloy Lafuente (stronk7) added a comment - - edited Ok, these have been tested and passed against the branches in previous comment: 1) default configuration, not using alternative caches: complete phpunit. 28_STABLE (ok) and master (ok) . behat (@core_course,@core_group). 28_STABLE (ok) and master (ok) . 2) configuration using memcached via TEST_CACHE_USING_APPLICATION_STORE complete phpunit. 28_STABLE (ok) and master (ok) . behat (@core_course,@core_group). 28_STABLE (ok) and master (ok) . 3) configuration using memcache via $CFG->altcacheconfigpath complete phpunit. 28_STABLE (ok) and master (ok) . behat (@core_course,@core_group). 28_STABLE (ok) and master (ok) .
            Hide
            Eloy Lafuente (stronk7) added a comment -

            Integrated (28 & master), thanks!

            Show
            Eloy Lafuente (stronk7) added a comment - Integrated (28 & master), thanks!
            Hide
            Eloy Lafuente (stronk7) added a comment -

            And passed, see 2 comments above, ciao

            Show
            Eloy Lafuente (stronk7) added a comment - And passed, see 2 comments above, ciao
            Hide
            Frédéric Massart added a comment -

            Failing, this causes the following error:

            [Tue Dec 23 11:10:19.138484 2014] [:error] [pid 29321] [client 127.0.0.1:58482] PHP Fatal error:  Call to undefined function behat_is_test_site() in /home/fred/www/repositories/im/moodle/cache/classes/factory.php on line 129
            [Tue Dec 23 11:10:19.138510 2014] [:error] [pid 29321] [client 127.0.0.1:58482] PHP Stack trace:
            [Tue Dec 23 11:10:19.138518 2014] [:error] [pid 29321] [client 127.0.0.1:58482] PHP   1. {main}() /home/fred/www/repositories/im/moodle/index.php:0
            [Tue Dec 23 11:10:19.138522 2014] [:error] [pid 29321] [client 127.0.0.1:58482] PHP   2. require_once() /home/fred/www/repositories/im/moodle/index.php:30
            [Tue Dec 23 11:10:19.138526 2014] [:error] [pid 29321] [client 127.0.0.1:58482] PHP   3. require_once() /home/fred/www/repositories/im/moodle/config.php:55
            [Tue Dec 23 11:10:19.138530 2014] [:error] [pid 29321] [client 127.0.0.1:58482] PHP   4. initialise_cfg() /home/fred/www/repositories/im/moodle/lib/setup.php:630
            [Tue Dec 23 11:10:19.138533 2014] [:error] [pid 29321] [client 127.0.0.1:58482] PHP   5. get_config() /home/fred/www/repositories/im/moodle/lib/setuplib.php:731
            [Tue Dec 23 11:10:19.138537 2014] [:error] [pid 29321] [client 127.0.0.1:58482] PHP   6. cache::make() /home/fred/www/repositories/im/moodle/lib/moodlelib.php:1456
            [Tue Dec 23 11:10:19.138541 2014] [:error] [pid 29321] [client 127.0.0.1:58482] PHP   7. cache_factory::instance() /home/fred/www/repositories/im/moodle/cache/classes/loaders.php:173
            

            I seem to be able to replicate this by removing any behat related $CFG values from my config.php.

            Show
            Frédéric Massart added a comment - Failing, this causes the following error: [Tue Dec 23 11:10:19.138484 2014] [:error] [pid 29321] [client 127.0.0.1:58482] PHP Fatal error: Call to undefined function behat_is_test_site() in /home/fred/www/repositories/im/moodle/cache/classes/factory.php on line 129 [Tue Dec 23 11:10:19.138510 2014] [:error] [pid 29321] [client 127.0.0.1:58482] PHP Stack trace: [Tue Dec 23 11:10:19.138518 2014] [:error] [pid 29321] [client 127.0.0.1:58482] PHP 1. {main}() /home/fred/www/repositories/im/moodle/index.php:0 [Tue Dec 23 11:10:19.138522 2014] [:error] [pid 29321] [client 127.0.0.1:58482] PHP 2. require_once() /home/fred/www/repositories/im/moodle/index.php:30 [Tue Dec 23 11:10:19.138526 2014] [:error] [pid 29321] [client 127.0.0.1:58482] PHP 3. require_once() /home/fred/www/repositories/im/moodle/config.php:55 [Tue Dec 23 11:10:19.138530 2014] [:error] [pid 29321] [client 127.0.0.1:58482] PHP 4. initialise_cfg() /home/fred/www/repositories/im/moodle/lib/setup.php:630 [Tue Dec 23 11:10:19.138533 2014] [:error] [pid 29321] [client 127.0.0.1:58482] PHP 5. get_config() /home/fred/www/repositories/im/moodle/lib/setuplib.php:731 [Tue Dec 23 11:10:19.138537 2014] [:error] [pid 29321] [client 127.0.0.1:58482] PHP 6. cache::make() /home/fred/www/repositories/im/moodle/lib/moodlelib.php:1456 [Tue Dec 23 11:10:19.138541 2014] [:error] [pid 29321] [client 127.0.0.1:58482] PHP 7. cache_factory::instance() /home/fred/www/repositories/im/moodle/cache/classes/loaders.php:173 I seem to be able to replicate this by removing any behat related $CFG values from my config.php.
            Hide
            David Mudrak added a comment -

            Similar error here when trying to access freshly installed MOODLE_28_STABLE from integration.git:

            Fatal error: Call to undefined function behat_is_test_site() in .../mdk/i28/moodle/cache/classes/factory.php on line 129
            

            The following obvious patch made the site usable to me:

            diff --git a/cache/classes/factory.php b/cache/classes/factory.php
            index fd78222..47cb456 100644
            --- a/cache/classes/factory.php
            +++ b/cache/classes/factory.php
            @@ -28,6 +28,8 @@
             
             defined('MOODLE_INTERNAL') || die();
             
            +require_once($CFG->libdir.'/behat/lib.php');
            +
             /**
              * The cache factory class.
              *
            

            Show
            David Mudrak added a comment - Similar error here when trying to access freshly installed MOODLE_28_STABLE from integration.git: Fatal error: Call to undefined function behat_is_test_site() in .../mdk/i28/moodle/cache/classes/factory.php on line 129 The following obvious patch made the site usable to me: diff --git a/cache/classes/factory.php b/cache/classes/factory.php index fd78222..47cb456 100644 --- a/cache/classes/factory.php +++ b/cache/classes/factory.php @@ -28,6 +28,8 @@ defined('MOODLE_INTERNAL') || die(); +require_once($CFG->libdir.'/behat/lib.php'); + /** * The cache factory class. *
            Hide
            Eloy Lafuente (stronk7) added a comment -

            Oh, thanks for spotting this guys!

            On it right now...

            Show
            Eloy Lafuente (stronk7) added a comment - Oh, thanks for spotting this guys! On it right now...
            Hide
            Eloy Lafuente (stronk7) added a comment -

            Aha, reproduced it here at first attempt to use the caches without behat variables defined. What I'm going to do is:

            1) Replace wrong use of behat_is_test_site() in core by if (defined('BEHAT_SITE_RUNNING')).
            2) Document that the function is not suitable for use in code, sort of private.

            Ciao

            Show
            Eloy Lafuente (stronk7) added a comment - Aha, reproduced it here at first attempt to use the caches without behat variables defined. What I'm going to do is: 1) Replace wrong use of behat_is_test_site() in core by if (defined('BEHAT_SITE_RUNNING')) . 2) Document that the function is not suitable for use in code, sort of private. Ciao
            Hide
            Eloy Lafuente (stronk7) added a comment -

            Proposal:

            https://github.com/stronk7/moodle/commit/c0d8831ae72319bdf1a6f6c84433af1b3bf87789

            Just re-running some install/upgrade/exploratory tests here...

            Show
            Eloy Lafuente (stronk7) added a comment - Proposal: https://github.com/stronk7/moodle/commit/c0d8831ae72319bdf1a6f6c84433af1b3bf87789 Just re-running some install/upgrade/exploratory tests here...
            Hide
            Eloy Lafuente (stronk7) added a comment -

            I've integrated the fix above both for 28_STABLE and master... after trying install/upgrade/unit tests with behat variables disabled in master. Repeating tests now against 28_STABLE.

            Show
            Eloy Lafuente (stronk7) added a comment - I've integrated the fix above both for 28_STABLE and master... after trying install/upgrade/unit tests with behat variables disabled in master. Repeating tests now against 28_STABLE.
            Hide
            Eloy Lafuente (stronk7) added a comment -

            Ok, I'm passing this. Have tried install & upgrade, unit tests, exploratory testing with default caches and also with memcache/mencached ones. And also setting again the behat vars and running some features. Everything went ok apparently.

            Show
            Eloy Lafuente (stronk7) added a comment - Ok, I'm passing this. Have tried install & upgrade, unit tests, exploratory testing with default caches and also with memcache/mencached ones. And also setting again the behat vars and running some features. Everything went ok apparently.
            Hide
            Dan Poltawski added a comment -

            You've made it into the Moodle Christmas present, it can be collected from git.moodle.org right now or https://download.moodle.org in some hours.

            On behalf of everyone at Moodle - Merry Christmas

            “Christmas doesn't come from a store, maybe Christmas perhaps means a little bit more....”
            ― Dr. Seuss, How the Grinch Stole Christmas!

            Show
            Dan Poltawski added a comment - You've made it into the Moodle Christmas present, it can be collected from git.moodle.org right now or https://download.moodle.org in some hours. On behalf of everyone at Moodle - Merry Christmas “Christmas doesn't come from a store, maybe Christmas perhaps means a little bit more....” ― Dr. Seuss, How the Grinch Stole Christmas!
            Hide
            David Monllaó added a comment -

            About Eloy's Or "perhaps we should consider the fact of the behat site not writing the alternative file a bug?" IMO should not be required to have a normal site running in wwwroot to make this work, as somebody said most of CI jobs running phpunit or behat probably don't have one.

            Show
            David Monllaó added a comment - About Eloy's Or "perhaps we should consider the fact of the behat site not writing the alternative file a bug?" IMO should not be required to have a normal site running in wwwroot to make this work, as somebody said most of CI jobs running phpunit or behat probably don't have one.
            Hide
            Eloy Lafuente (stronk7) added a comment -

            Would you mind creating an issue about that, then?

            Show
            Eloy Lafuente (stronk7) added a comment - Would you mind creating an issue about that, then?

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: