Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-51584

Regrading a large course hits the cache so many times that it slows down processing

XMLWordPrintable

    • MOODLE_28_STABLE
    • MOODLE_30_STABLE, MOODLE_31_STABLE
    • Hide
      Observing the performance change

      First, creating a test course:

      • Generate a large couse with the following command:

        php admin/tool/generator/cli/maketestcourse.php --shortname=GRADEBOOKM --size=M --fixeddataset --filesizelimit=1024

      • For Assignment 1, grade ~300 users. I did this by doing the following:
        • Go to Assignment 1
        • Enable quick grading and change number of items to show per page to 100.
        • Pop open the browser console and enter the following and hit return (it will enter random grades for you):

          Y.all('.quickgradingform input[type=text]').each(function(n) { n.setAttribute('value', Math.floor(Math.random() * 101)) });

        • Click on Save changes. Repeat for the next 2 pages.
      • Go into the gradebook.
      • Add a new Category, I used Natural for the aggregation method.

      Be sure to enable performance info in the footer so you can see cache hits.

      Trigger a regrade and note performance. The way I was doing this was with the following:

      • Go to the gradebook, and then Setup
      • Take note of the cache hits for core/config - static acceleration.
      • Move Assignment 1 into the newly created category
      • Take note of the increase in cache hits for core/config - static acceleration.
      Other testing

      Be sure to run ALL PHPUnit tests and Behat features. Make sure installation (cli and web) work fine and upgrades are OK as well. Basically please throw all the automated testing at it!

      Show
      Observing the performance change First, creating a test course: Generate a large couse with the following command: php admin/tool/generator/cli/maketestcourse.php --shortname=GRADEBOOKM --size=M --fixeddataset --filesizelimit=1024 For Assignment 1, grade ~300 users. I did this by doing the following: Go to Assignment 1 Enable quick grading and change number of items to show per page to 100. Pop open the browser console and enter the following and hit return (it will enter random grades for you): Y.all('.quickgradingform input[type=text]').each(function(n) { n.setAttribute('value', Math.floor(Math.random() * 101)) }); Click on Save changes. Repeat for the next 2 pages. Go into the gradebook. Add a new Category, I used Natural for the aggregation method. Be sure to enable performance info in the footer so you can see cache hits. Trigger a regrade and note performance. The way I was doing this was with the following: Go to the gradebook, and then Setup Take note of the cache hits for core/config - static acceleration. Move Assignment 1 into the newly created category Take note of the increase in cache hits for core/config - static acceleration. Other testing Be sure to run ALL PHPUnit tests and Behat features. Make sure installation (cli and web) work fine and upgrades are OK as well. Basically please throw all the automated testing at it!

      With very large courses (EG: thousands of students and hundreds of graded activities), the cost to regrade the gradebook is very high. While profiling this for MDL-51498, found that grade_grade->get_grade_min_and_max is pretty expensive and it mostly boils down to it pounding the cache with get_config calls. On a small generated course, get_config is called 2906 times in total, 2600 times directly from grade_grade->get_grade_min_and_max.

        1. callgraph.svg
          30 kB
          Mark Nielsen
        2. get_config.patch
          0.7 kB
          Mark Nielsen
        3. minmax.patch
          1 kB
          Mark Nielsen

            mr-russ Russell Smith
            bushido Mark Nielsen
            Andrew Lyons Andrew Lyons
            Dan Poltawski Dan Poltawski
            Adrian Greeve Adrian Greeve
            Votes:
            4 Vote for this issue
            Watchers:
            17 Start watching this issue

              Created:
              Updated:
              Resolved:

                Error rendering 'clockify-timesheets-time-tracking-reports:timer-sidebar'. Please contact your Jira administrators.