Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-81456

Serious performance regression in phpunit

XMLWordPrintable

      A massive spike in PHPUnit time runs has been observed between commits f3f514ace1931a8e446285f74e36de199cc14497 and f150d5016de229bac45c54f1a3a36b2a94ee95d0.

      This can be seen very clearly on the following graph.


      See https://ci.moodle.org/view/B%20-%20main/job/W.01.01%20-%20PHPUnit%20-%20Postgres/buildTimeTrend for the build graph trend.

      The issues in this span are the following, and their times to run our standard SQL Server PHPUnit Pre-integration jobs are as follows:

      81132 - 81132 - 1:25
      81313 - 81313 - 1:25
      80843 - 80843 - 1:19
      80099 - 80099 - 2:37
      81319 - 81319 - 1:21
      81023 - 81023 - 1:23
      80715 - 80715 - 1:21

      Clearly MDL-80099 is the most obvious candidate.

      Reverting this commit in particular significantly reduces the time: 788556f652e8efe28e46c1c2275e636c06aeaae0.

      Here's an example from current origin/maiun:

      ➜  im git:(main) ./vendor/bin/phpunit lib/tests/weblib_test.php
      Moodle 4.4dev+ (Build: 20240405), 26649f5750ee3d35e56990512ee4c79d388cea8b
      Php: 8.2.11, pgsql: 14.9 (Homebrew), OS: Darwin 23.2.0 arm64
      PHPUnit 9.6.18 by Sebastian Bergmann and contributors.
       
      ...............................................................  63 / 118 ( 53%)
      .......................................................         118 / 118 (100%)
       
      Time: 01:08.428, Memory: 76.50 MB
       
      OK (118 tests, 347 assertions)
      

      Versus after reverging that commit:

      ➜  im git:(main) ✗ ./vendor/bin/phpunit lib/tests/weblib_test.php
      Moodle 4.4dev+ (Build: 20240405), 26649f5750ee3d35e56990512ee4c79d388cea8b
      Php: 8.2.11, pgsql: 14.9 (Homebrew), OS: Darwin 23.2.0 arm64
      PHPUnit 9.6.18 by Sebastian Bergmann and contributors.
       
      ...............................................................  63 / 118 ( 53%)
      .......................................................         118 / 118 (100%)
       
      Time: 00:07.118, Memory: 76.50 MB
       
      OK (118 tests, 347 assertions)
      

      As you can see, the test time for this run file is approximately 10 times faster on this one test.

      This is a critical must fix and requires our immediate attention as it blocks release.

            dobedobedoh Andrew Lyons
            dobedobedoh Andrew Lyons
            Huong Nguyen Huong Nguyen
            Sara Arjona (@sarjona) Sara Arjona (@sarjona)
            CiBoT CiBoT
            Votes:
            1 Vote for this issue
            Watchers:
            9 Start watching this issue

              Created:
              Updated:
              Resolved:

                Estimated:
                Original Estimate - 0 minutes
                0m
                Remaining:
                Remaining Estimate - 0 minutes
                0m
                Logged:
                Time Spent - 1 day, 2 minutes
                1d 2m

                  Error rendering 'clockify-timesheets-time-tracking-reports:timer-sidebar'. Please contact your Jira administrators.