[MDL-69271] Random phpunit failure with auth_ldap_plugin_testcase::test_auth_ldap - Moodle Tracker

XML

Word

Printable

Details

Type: Bug
Status: Closed
Priority: Minor
Resolution: Fixed
Affects Version/s: 3.8.4, 3.9.1, 3.10
Fix Version/s: 3.5.14, 3.7.8, 3.8.5, 3.9.2
Component/s: Authentication, Unit tests
Labels:
- ci
- triaged

Testing Instructions:

Hide

Requirements.

ldap service available and phpunit configured to use it.

Before patch (to reproduce the problem).

Edit the auth/ldap/auth.php file and perform this change:

diff --git a/auth/ldap/auth.php b/auth/ldap/auth.php

index f03372f3154..a65269ef8ca 100644

--- a/auth/ldap/auth.php

+++ b/auth/ldap/auth.php

@@ -925,7 +925,7 @@ class auth_plugin_ldap extends auth_plugin_base {

         $sql = 'SELECT e.id, e.username

                   FROM {tmp_extuser} e

                   LEFT JOIN {user} u ON (e.username = u.username AND e.mnethostid = u.mnethostid)

-                 WHERE u.id IS NULL';

+                 WHERE u.id IS NULL ORDER BY e.username DESC';

         $add_users = $DB->get_records_sql($sql);

         if (!empty($add_users)) {

Init phpunit: php admin/tool/phpunit/cli/init.php.
Run vendor/bin/phpunit --filter 'test_auth_ldap'

Verify that you get 8 errors like this:

Failed asserting that core\event\user_created Object (...) is an instance of class "\core\event\role_assigned".

After patch (to verify the fix).

Repeat steps 1-3 from previous section.

Verify that the test ends OK, without errors (that means the test is now immune to any ordering, implicit or explicit):

Moodle 4.0dev (Build: 20200723), d4e8743a133752123aabfaa514c7e3e18c56baca

Php: 7.3.16, pgsql: 9.6.17, OS: Darwin 19.6.0 x86_64

PHPUnit 7.5.20 by Sebastian Bergmann and contributors.

........                                                            8 / 8 (100%)

Time: 9 seconds, Memory: 465.00 MB

OK (8 tests, 472 assertions)

Show

Requirements. ldap service available and phpunit configured to use it. Before patch (to reproduce the problem). Edit the auth/ldap/auth.php file and perform this change: diff --git a/auth/ldap/auth.php b/auth/ldap/auth.php index f03372f3154..a65269ef8ca 100644 --- a/auth/ldap/auth.php +++ b/auth/ldap/auth.php @@ -925,7 +925,7 @@ class auth_plugin_ldap extends auth_plugin_base { $sql = 'SELECT e.id, e.username FROM {tmp_extuser} e LEFT JOIN {user} u ON (e.username = u.username AND e.mnethostid = u.mnethostid) - WHERE u.id IS NULL'; + WHERE u.id IS NULL ORDER BY e.username DESC'; $add_users = $DB->get_records_sql($sql); if (!empty($add_users)) { Init phpunit: php admin/tool/phpunit/cli/init.php . Run vendor/bin/phpunit --filter 'test_auth_ldap' Verify that you get 8 errors like this: Failed asserting that core\event\user_created Object (...) is an instance of class "\core\event\role_assigned". After patch (to verify the fix). Repeat steps 1-3 from previous section. Verify that the test ends OK, without errors (that means the test is now immune to any ordering, implicit or explicit): Moodle 4.0dev (Build: 20200723), d4e8743a133752123aabfaa514c7e3e18c56baca Php: 7.3.16, pgsql: 9.6.17, OS: Darwin 19.6.0 x86_64 PHPUnit 7.5.20 by Sebastian Bergmann and contributors. ........ 8 / 8 (100%) Time: 9 seconds, Memory: 465.00 MB OK (8 tests, 472 assertions)

Affected Branches:
MOODLE_310_STABLE, MOODLE_38_STABLE, MOODLE_39_STABLE
Fixed Branches:
MOODLE_35_STABLE, MOODLE_37_STABLE, MOODLE_38_STABLE, MOODLE_39_STABLE
Epic Link:
CI random failures
Pull from Repository:
https://github.com/stronk7/moodle.git
Pull 3.5 Branch:
~~MDL-69271~~_35
Pull 3.5 Diff URL:
https://github.com/stronk7/moodle/compare/MDL-69271_35~1...MDL-69271_35
Pull 3.8 Branch:
~~MDL-69271~~_38
Pull 3.8 Diff URL:
https://github.com/stronk7/moodle/compare/MDL-69271_38~1...MDL-69271_38
Pull 3.9 Branch:
~~MDL-69271~~_39
Pull 3.9 Diff URL:
https://github.com/stronk7/moodle/compare/MDL-69271_39~1...MDL-69271_39
Pull Master Branch:
~~MDL-69271~~
Pull Master Diff URL:
https://github.com/stronk7/moodle/compare/MDL-69271~1...MDL-69271

Description

It has started to happen recently (few weeks before the creation of this issue).

Link: https://ci.moodle.org/job/W39.01.05%20-%20PHPUnit%20-%20Oracle%20(php73)/11/

There were 8 failures:

1) auth_ldap_plugin_testcase::test_auth_ldap with data set "pagesize 1, subcontexts 0" (1, 0)

Failed asserting that core\event\user_created Object (...) is an instance of class "\core\event\role_assigned".

/var/www/html/auth/ldap/tests/plugin_test.php:183

/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:

 vendor/bin/phpunit --verbose "auth_ldap_plugin_testcase" auth/ldap/tests/plugin_test.php

2) auth_ldap_plugin_testcase::test_auth_ldap with data set "pagesize 1, subcontexts 1" (1, 1)

Failed asserting that core\event\user_created Object (...) is an instance of class "\core\event\role_assigned".

/var/www/html/auth/ldap/tests/plugin_test.php:183

/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:

 vendor/bin/phpunit --verbose "auth_ldap_plugin_testcase" auth/ldap/tests/plugin_test.php

3) auth_ldap_plugin_testcase::test_auth_ldap with data set "pagesize 3, subcontexts 0" (3, 0)

Failed asserting that core\event\user_created Object (...) is an instance of class "\core\event\role_assigned".

/var/www/html/auth/ldap/tests/plugin_test.php:183

/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:

 vendor/bin/phpunit --verbose "auth_ldap_plugin_testcase" auth/ldap/tests/plugin_test.php

4) auth_ldap_plugin_testcase::test_auth_ldap with data set "pagesize 3, subcontexts 1" (3, 1)

Failed asserting that core\event\user_created Object (...) is an instance of class "\core\event\role_assigned".

/var/www/html/auth/ldap/tests/plugin_test.php:183

/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:

 vendor/bin/phpunit --verbose "auth_ldap_plugin_testcase" auth/ldap/tests/plugin_test.php

5) auth_ldap_plugin_testcase::test_auth_ldap with data set "pagesize 5, subcontexts 0" (5, 0)

Failed asserting that core\event\user_created Object (...) is an instance of class "\core\event\role_assigned".

/var/www/html/auth/ldap/tests/plugin_test.php:183

/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:

 vendor/bin/phpunit --verbose "auth_ldap_plugin_testcase" auth/ldap/tests/plugin_test.php

6) auth_ldap_plugin_testcase::test_auth_ldap with data set "pagesize 5, subcontexts 1" (5, 1)

Failed asserting that core\event\user_created Object (...) is an instance of class "\core\event\role_assigned".

/var/www/html/auth/ldap/tests/plugin_test.php:183

/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:

 vendor/bin/phpunit --verbose "auth_ldap_plugin_testcase" auth/ldap/tests/plugin_test.php

7) auth_ldap_plugin_testcase::test_auth_ldap with data set "pagesize 1000, subcontexts 0" (1000, 0)

Failed asserting that core\event\user_created Object (...) is an instance of class "\core\event\role_assigned".

/var/www/html/auth/ldap/tests/plugin_test.php:183

/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:

 vendor/bin/phpunit --verbose "auth_ldap_plugin_testcase" auth/ldap/tests/plugin_test.php

8) auth_ldap_plugin_testcase::test_auth_ldap with data set "pagesize 1000, subcontexts 1" (1000, 1)

Failed asserting that core\event\user_created Object (...) is an instance of class "\core\event\role_assigned".

/var/www/html/auth/ldap/tests/plugin_test.php:183

/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

To re-run:

 vendor/bin/phpunit --verbose "auth_ldap_plugin_testcase" auth/ldap/tests/plugin_test.php

Seems to happen with Oracle (slowness?) and not with others. Detected in 38, 39 and master.

Attachments

Activity

People

Assignee:: Eloy Lafuente (stronk7)

Reporter:: Eloy Lafuente (stronk7)

Peer reviewer:: Jake Dallimore

Integrator:: Andrew Nicols

Tester:: Andrew Nicols

Participants:: Andrew Nicols, CiBoT, Eloy Lafuente (stronk7), Jake Dallimore

Component watchers:: Adrian Greeve, Jake Dallimore, Mathew May, Mihail Geshoski, Peter Dias, Sujith Haridasan, Andrew Nicols, Jun Pataleta, Michael Hawkins, Shamim Rezaie, Simey Lameze

Votes:: 0 Vote for this issue

Watchers:: 3 Start watching this issue

Dates

Created:: 14/Jul/20 6:39 PM

Updated:: 22/Aug/20 3:11 AM

Resolved:: 22/Aug/20 3:11 AM

Fix Release Date:: 14/Sep/20

Time Tracking

Estimated:

Not Specified

Remaining:

Logged:

2h 22m