Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-75708

Only run test_stress_datasource for PHPUNIT_LONGTEST

    XMLWordPrintable

Details

    • MOODLE_400_STABLE
    • MOODLE_400_STABLE
    • Hide
      1. Confirm the above GHA runs both pass ☝️
      2. Run following unit test:

        $ bin/moodle-docker-compose exec webserver vendor/bin/phpunit --verbose admin/tests/reportbuilder/datasource/task_logs_test.php
        

      3. Confirm you see the following output:

        There was 1 skipped test:
         
        1) core_admin\reportbuilder\datasource\task_logs_test::test_stress_datasource
        PHPUNIT_LONGTEST is not defined
         
        /var/www/html/admin/tests/reportbuilder/datasource/task_logs_test.php:231
        /var/www/html/lib/phpunit/classes/advanced_testcase.php:80
         
        OK, but incomplete, skipped, or risky tests!
        Tests: 23, Assertions: 37, Skipped: 1.
        

      4. Edit site config.php, add the following line:

        define('PHPUNIT_LONGTEST', true);
        

      5. Re-run same unit test
      6. Confirm you see the following output (no skipped tests):

        OK (23 tests, 290 assertions)
        

      Show
      Confirm the above GHA runs both pass ☝️ Run following unit test: $ bin/moodle-docker-compose exec webserver vendor/bin/phpunit --verbose admin/tests/reportbuilder/datasource/task_logs_test.php Confirm you see the following output: There was 1 skipped test:   1) core_admin\reportbuilder\datasource\task_logs_test::test_stress_datasource PHPUNIT_LONGTEST is not defined   /var/www/html/admin/tests/reportbuilder/datasource/task_logs_test.php:231 /var/www/html/lib/phpunit/classes/advanced_testcase.php:80   OK, but incomplete, skipped, or risky tests! Tests: 23, Assertions: 37, Skipped: 1. Edit site config.php , add the following line: define('PHPUNIT_LONGTEST', true); Re-run same unit test Confirm you see the following output (no skipped tests): OK (23 tests, 290 assertions)

    Description

      We've started facing the following failures in GHA:

      There were 2 failures:
      1) core_badges\reportbuilder\datasource\badges_test::test_stress_datasource
      Error for column 'badge:name' with aggregation 'groupconcatdistinct': mysqli::query(): (HY001/1038): Out of sort memory, consider increasing server sort buffer size
      /home/runner/work/moodle/moodle/reportbuilder/tests/helpers.php:124
      /home/runner/work/moodle/moodle/badges/tests/reportbuilder/datasource/badges_test.php:113
      /home/runner/work/moodle/moodle/lib/phpunit/classes/advanced_testcase.php:80
      phpvfscomposer:///home/runner/work/moodle/moodle/vendor/phpunit/phpunit/phpunit:97
      2) core_course\reportbuilder\datasource\participants_test::test_stress_datasource
      Error for column 'course:coursefullnamewithlink' with aggregation 'groupconcatdistinct': mysqli::query(): (HY001/1038): Out of sort memory, consider increasing server sort buffer size
      /home/runner/work/moodle/moodle/reportbuilder/tests/helpers.php:124
      /home/runner/work/moodle/moodle/course/tests/reportbuilder/datasource/participants_test.php:291
      /home/runner/work/moodle/moodle/lib/phpunit/classes/advanced_testcase.php:80
      phpvfscomposer:///home/runner/work/moodle/moodle/vendor/phpunit/phpunit/phpunit:97
      

       

      It looks like some MySQL / MariaDB setting needs to be increased. The possibilities to deal with it are:

      1. Try to reduce the amount of data the tests use (while keeping them still stress-useful).
      2. Avoid running them by default.
      3. Modify GHA / Travis settings

      After discussing it with Paul and the iTeam, we've decided to go for #2 and add these tests to PHPUNIT_LONGTEST (so they won't be executed in GHA/Travis). That's OK because Eloy has confirmed that we run CIs with them enabled: https://github.com/moodlehq/moodle-ci-runner/blob/master/runner/master/config.template.php#L76

       

      This should be applied to all the report source stress tests (king of as policy), so we are covered if they increase in the future  

      Attachments

        Issue Links

          Activity

            People

              pholden Paul Holden
              sarjona Sara Arjona (@sarjona)
              Sara Arjona (@sarjona) Sara Arjona (@sarjona)
              Jun Pataleta Jun Pataleta
              Jun Pataleta Jun Pataleta
              Votes:
              2 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 2 hours, 26 minutes
                  2h 26m

                  Clockify

                    Error rendering 'clockify-timesheets-time-tracking-reports:timer-sidebar'. Please contact your Jira administrators.