[MDL-75708] Only run test_stress_datasource for PHPUNIT_LONGTEST - Moodle Tracker

XML

Word

Printable

Details

Type: Bug
Status: Closed
Priority: Minor
Resolution: Fixed
Affects Version/s: 4.0.3
Fix Version/s: 4.0.4
Component/s: Report builder, Unit tests
Labels:
- ci
- triaged

Affected Branches:
MOODLE_400_STABLE
Fixed Branches:
MOODLE_400_STABLE
Pull from Repository:
https://github.com/paulholden/moodle.git
Pull 4.0 Branch:
~~MDL-75708~~-400
Pull 4.0 Diff URL:
https://github.com/moodle/moodle/compare/MOODLE_400_STABLE...paulholden:MDL-75708-400
Pull Master Branch:
~~MDL-75708~~
Pull Master Diff URL:
https://github.com/moodle/moodle/compare/master...paulholden:MDL-75708

Testing Instructions:

Hide

Confirm the above GHA runs both pass ☝️

Run following unit test:

$ bin/moodle-docker-compose exec webserver vendor/bin/phpunit --verbose admin/tests/reportbuilder/datasource/task_logs_test.php

Confirm you see the following output:

There was 1 skipped test:

1) core_admin\reportbuilder\datasource\task_logs_test::test_stress_datasource

PHPUNIT_LONGTEST is not defined

/var/www/html/admin/tests/reportbuilder/datasource/task_logs_test.php:231

/var/www/html/lib/phpunit/classes/advanced_testcase.php:80

OK, but incomplete, skipped, or risky tests!

Tests: 23, Assertions: 37, Skipped: 1.

Edit site config.php, add the following line:
define('PHPUNIT_LONGTEST', true);
Re-run same unit test
Confirm you see the following output (no skipped tests):
OK (23 tests, 290 assertions)

Show

Confirm the above GHA runs both pass ☝️ Run following unit test: $ bin/moodle-docker-compose exec webserver vendor/bin/phpunit --verbose admin/tests/reportbuilder/datasource/task_logs_test.php Confirm you see the following output: There was 1 skipped test: 1) core_admin\reportbuilder\datasource\task_logs_test::test_stress_datasource PHPUNIT_LONGTEST is not defined /var/www/html/admin/tests/reportbuilder/datasource/task_logs_test.php:231 /var/www/html/lib/phpunit/classes/advanced_testcase.php:80 OK, but incomplete, skipped, or risky tests! Tests: 23, Assertions: 37, Skipped: 1. Edit site config.php , add the following line: define('PHPUNIT_LONGTEST', true); Re-run same unit test Confirm you see the following output (no skipped tests): OK (23 tests, 290 assertions)

Description

We've started facing the following failures in GHA:

There were 2 failures:

1) core_badges\reportbuilder\datasource\badges_test::test_stress_datasource

Error for column 'badge:name' with aggregation 'groupconcatdistinct': mysqli::query(): (HY001/1038): Out of sort memory, consider increasing server sort buffer size

/home/runner/work/moodle/moodle/reportbuilder/tests/helpers.php:124

/home/runner/work/moodle/moodle/badges/tests/reportbuilder/datasource/badges_test.php:113

/home/runner/work/moodle/moodle/lib/phpunit/classes/advanced_testcase.php:80

phpvfscomposer:///home/runner/work/moodle/moodle/vendor/phpunit/phpunit/phpunit:97

2) core_course\reportbuilder\datasource\participants_test::test_stress_datasource

Error for column 'course:coursefullnamewithlink' with aggregation 'groupconcatdistinct': mysqli::query(): (HY001/1038): Out of sort memory, consider increasing server sort buffer size

/home/runner/work/moodle/moodle/reportbuilder/tests/helpers.php:124

/home/runner/work/moodle/moodle/course/tests/reportbuilder/datasource/participants_test.php:291

/home/runner/work/moodle/moodle/lib/phpunit/classes/advanced_testcase.php:80

phpvfscomposer:///home/runner/work/moodle/moodle/vendor/phpunit/phpunit/phpunit:97

It looks like some MySQL / MariaDB setting needs to be increased. The possibilities to deal with it are:

Try to reduce the amount of data the tests use (while keeping them still stress-useful).
Avoid running them by default.
Modify GHA / Travis settings

After discussing it with Paul and the iTeam, we've decided to go for #2 and add these tests to PHPUNIT_LONGTEST (so they won't be executed in GHA/Travis). That's OK because Eloy has confirmed that we run CIs with them enabled: https://github.com/moodlehq/moodle-ci-runner/blob/master/runner/master/config.template.php#L76

This should be applied to all the report source stress tests (king of as policy), so we are covered if they increase in the future

Attachments

Issue Links

is a regression caused by

MDL-73938 Implement thorough report source testing ("stress" tests)

Closed

Activity

People

Assignee:: Paul Holden

Reporter:: Sara Arjona (@sarjona)

Peer reviewer:: Sara Arjona (@sarjona)

Integrator:: Jun Pataleta

Tester:: Jun Pataleta

Participants:: CiBoT, Jun Pataleta, Paul Holden, Sara Arjona (@sarjona), ToBiC

Component watchers:: Carlos Castillo, David Matamoros, Mikel Martín Corrales, Paul Holden

Votes:: 2 Vote for this issue

Watchers:: 7 Start watching this issue

Dates

Created:: 09/Sep/22 5:25 PM

Updated:: 10/Sep/22 4:34 PM

Resolved:: 10/Sep/22 4:34 PM

Fix Release Date:: 12/Sep/22

Time Tracking

Estimated:

Not Specified

Remaining:

Logged:

2h 26m