Affects Version/s: 2.8.1, 2.9
Component/s: Automated functional tests (behat)
- Open error log in one terminal to ensure you can see php errors
- Add @test tags to
- Modify Course 1 to Course 2 in mod/quiz/tests/behat/editing_set_marks_with_attempts.feature
- Run behat with tags @test
- Repeat above few times, ensuring you don't see any error in error log.
Pre-requisite: Open error log in one terminal to ensure you can see php errors Test 1: Run individual scenarios, ensuring you don't see any db error. Add @test tags to blocks/activity_modules/tests/behat/block_activity_modules.feature mod/quiz/tests/behat/editing_set_marks_with_attempts.feature Modify Course 1 to Course 2 in mod/quiz/tests/behat/editing_set_marks_with_attempts.feature Run behat with tags @test Repeat above few times, ensuring you don't see any error in error log. Test 2: Run whole suite Run whole suite ensuring you don't see any error related to dataroot dir/file creation (see comments for exact error)
- Run whole suite ensuring you don't see any error related to dataroot dir/file creation (see comments for exact error)
Affected Branches:MOODLE_28_STABLE, MOODLE_29_STABLE
Fixed Branches:MOODLE_27_STABLE, MOODLE_28_STABLE
Pull from Repository:
Pull Master Branch:wip-mdl-48302
Pull Master Diff URL:
This is a followup of
MDL-48251, where a "mysterious" interdependency between 2 tests was discovered (and fixed with a "nonsense" fix).
In any case, it demoed that, at some point, there are unwanted dependencies between tests, so they are not running 100% isolated.
This is about to detect what is causing isolation to fail and fix it. Note that it also may affect unit tests (if the problem is in some part of the "reset between tests" code).
To be able to reproduce the problem (only tried with Postgres here):
1. In master revert this (the "nonsense") commit: 7b9fb26
2. Edit "blocks/activity_modules/tests/behat/block_activity_modules.feature" and add the @isolated tag to it.
3. Edit "mod/quiz/tests/behat/editing_set_marks_with_attempts.feature" and add the @isolated tag to it.
4. Run behat with --tags='@isolated'
5. Failure happens and execution runs forever (it does not if the commit in point #1 is not reverted).
Posible points to review can be (note this is just a bet):
1) Some request/session/static MUC cache not being reset.
2) Some problem in generators incorrectly using static data.
3) Some problem with transactions, not properly closed or rolled-back incorrectly..
It would be great to, apart from the solution, provide some tests ensuring it does not happen anymore (behat or phpunit).
Note: I've set only 2.8.1 and 2.9 as affected versions (to match
MDL-48251)... but once discovered the culprit this surely should be extended to all supported versions.