Details
-
Type:
Improvement
-
Status: Open
-
Priority:
Minor
-
Resolution: Unresolved
-
Affects Version/s: 3.3.6, 3.4.3, 3.5
-
Fix Version/s: None
-
Component/s: Automated functional tests (behat), Unit tests
-
Labels:
-
Affected Branches:MOODLE_33_STABLE, MOODLE_34_STABLE, MOODLE_35_STABLE
Description
This is a similar in philosophy to -- which was done in Moodle 2.8. There, we randomised the starting value of DB sequences, to ensure that people did not write fragile unit tests because they had hard-coded expected ids in asserts.MDL-43835
There is another problem, where core unit tests assume that, for example, a new Moodle install only has one course category, but a third-party plugin my create a course category for its own use on install.
In order to prevent these test-breaking assumptions from getting in to Moodle core, perhaps we should change the PHPunit site installer, so that after a default Moodle install, it generates a small random number (0, 1 or 2) of some key entities (e.g. user, course, category, question). These things should be given very specific names, to minimise the change they break existing tests.
As a specific example, if you install qtype_coderunner, then core_question_privacy_provider_testcase starts failing.