Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-75708

Only run test_stress_datasource for PHPUNIT_LONGTEST

XMLWordPrintable

    • MOODLE_400_STABLE
    • MOODLE_400_STABLE
    • Hide
      1. Confirm the above GHA runs both pass ☝️
      2. Run following unit test: 

        $ bin/moodle-docker-compose exec webserver vendor/bin/phpunit --verbose admin/tests/reportbuilder/datasource/task_logs_test.php

      3. Confirm you see the following output: 

        There was 1 skipped test:
        		 
         
        		 
        1) core_admin\reportbuilder\datasource\task_logs_test::test_stress_datasource
        		 
        PHPUNIT_LONGTEST is not defined
        		 
         
        		 
        /var/www/html/admin/tests/reportbuilder/datasource/task_logs_test.php:231
        		 
        /var/www/html/lib/phpunit/classes/advanced_testcase.php:80
        		 
         
        		 
        OK, but incomplete, skipped, or risky tests!
        		 
        Tests: 23, Assertions: 37, Skipped: 1.

      4. Edit site config.php, add the following line: 

        define('PHPUNIT_LONGTEST', true);

      5. Re-run same unit test
      6. Confirm you see the following output (no skipped tests): 

        OK (23 tests, 290 assertions)

      Show
      Confirm the above GHA runs both pass ☝️ Run following unit test: $ bin/moodle-docker-compose exec webserver vendor/bin/phpunit --verbose admin/tests/reportbuilder/datasource/task_logs_test.php Confirm you see the following output: There was 1 skipped test:   1) core_admin\reportbuilder\datasource\task_logs_test::test_stress_datasource PHPUNIT_LONGTEST is not defined   /var/www/html/admin/tests/reportbuilder/datasource/task_logs_test.php:231 /var/www/html/lib/phpunit/classes/advanced_testcase.php:80   OK, but incomplete, skipped, or risky tests! Tests: 23, Assertions: 37, Skipped: 1. Edit site config.php , add the following line: define('PHPUNIT_LONGTEST', true); Re-run same unit test Confirm you see the following output (no skipped tests): OK (23 tests, 290 assertions)

      We've started facing the following failures in GHA:

      There were 2 failures:
      		 
      1) core_badges\reportbuilder\datasource\badges_test::test_stress_datasource
      		 
      Error for column 'badge:name' with aggregation 'groupconcatdistinct': mysqli::query(): (HY001/1038): Out of sort memory, consider increasing server sort buffer size
      		 
      /home/runner/work/moodle/moodle/reportbuilder/tests/helpers.php:124
      		 
      /home/runner/work/moodle/moodle/badges/tests/reportbuilder/datasource/badges_test.php:113
      		 
      /home/runner/work/moodle/moodle/lib/phpunit/classes/advanced_testcase.php:80
      		 
      phpvfscomposer:///home/runner/work/moodle/moodle/vendor/phpunit/phpunit/phpunit:97
      		 
      2) core_course\reportbuilder\datasource\participants_test::test_stress_datasource
      		 
      Error for column 'course:coursefullnamewithlink' with aggregation 'groupconcatdistinct': mysqli::query(): (HY001/1038): Out of sort memory, consider increasing server sort buffer size
      		 
      /home/runner/work/moodle/moodle/reportbuilder/tests/helpers.php:124
      		 
      /home/runner/work/moodle/moodle/course/tests/reportbuilder/datasource/participants_test.php:291
      		 
      /home/runner/work/moodle/moodle/lib/phpunit/classes/advanced_testcase.php:80
      		 
      phpvfscomposer:///home/runner/work/moodle/moodle/vendor/phpunit/phpunit/phpunit:97

       

      It looks like some MySQL / MariaDB setting needs to be increased. The possibilities to deal with it are:

      1. Try to reduce the amount of data the tests use (while keeping them still stress-useful).
      2. Avoid running them by default.
      3. Modify GHA / Travis settings

      After discussing it with Paul and the iTeam, we've decided to go for #2 and add these tests to PHPUNIT_LONGTEST (so they won't be executed in GHA/Travis). That's OK because Eloy has confirmed that we run CIs with them enabled: https://github.com/moodlehq/moodle-ci-runner/blob/master/runner/master/config.template.php#L76

       

      This should be applied to all the report source stress tests (king of as policy), so we are covered if they increase in the future  

            pholden Paul Holden
            sarjona Sara Arjona (@sarjona)
            Sara Arjona (@sarjona) Sara Arjona (@sarjona)
            Jun Pataleta Jun Pataleta
            Jun Pataleta Jun Pataleta
            Votes:
            2 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved:

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0 minutes
                0m
                Logged:
                Time Spent - 2 hours, 26 minutes
                2h 26m

                  Error rendering 'clockify-timesheets-time-tracking-reports:timer-sidebar'. Please contact your Jira administrators.