Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-67674

Performance: Course category tree cache can get built in parallel

XMLWordPrintable

    • MOODLE_37_STABLE, MOODLE_38_STABLE, MOODLE_39_STABLE
    • MOODLE_37_STABLE, MOODLE_38_STABLE
    • MDL-67674_master
    • Hide

      Note, this requires a small code change to simulate a slow rebuild of the coursecattree cache.

      • As admin, log in to your test site and go to the Site administration / Courses / Manage courses and categories
      • Open a second browser, log in as a second user account with the moodle/core:viewcourselist capability, and go to the course list (course/index.php)
      • Make a code change (just for testing) to course/classes/category.php:
        1. Search for the comment: // Re-build the tree.
        2. Immediately above this line, add the following line: sleep(10); debugging('Actually building course category tree');
      • Using the admin browser, click the up or down arrow on a category to change the category order (changing the order will invalidate the cache). Once this as completed, reload the page.
      • Using the other (user) browser, reload the current page. (You need to do this step within ten seconds - ideally a couple of seconds after reloading the admin browser.)
        • EXPECTED: The admin browser takes 10 seconds then displays debugging info about rebuilding the course category tree.
        • EXPECTED: The other browser finishes slightly faster - it should load immediately after the admin browser does. There should be no debugging info.
      • In the admin browser, open a new tab and go to Site administration / Plugins / Caching / Configuration.
      • Find the coursecattree cache area, and click Purge next to it.
      • Wait for this to finish.
      • In the user browser, reload the page.
      • In the admin browser, reload the Manage courses and categories tab.
        • EXPECTED: The user browser takes 10 seconds then displays the page, including debugging info about rebuilding the course cache.
        • EXPECTED: The admin browser finishes immediately after the user browser does, and there should be no debugging info.
      Show
      Note, this requires a small code change to simulate a slow rebuild of the coursecattree cache. As admin, log in to your test site and go to the Site administration / Courses / Manage courses and categories Open a second browser, log in as a second user account with the moodle/core:viewcourselist capability, and go to the course list ( course/index.php ) Make a code change (just for testing) to course/classes/category.php: Search for the comment:  // Re-build the tree. Immediately above this line, add the following line: sleep(10); debugging('Actually building course category tree'); Using the admin browser, click the up or down arrow on a category to change the category order (changing the order will invalidate the cache). Once this as completed, reload the page. Using the other (user) browser, reload the current page. (You need to do this step within ten seconds - ideally a couple of seconds after reloading the admin browser.) EXPECTED: The admin browser takes 10 seconds then displays debugging info about rebuilding the course category tree. EXPECTED: The other browser finishes slightly faster - it should load immediately after the admin browser does. There should be no debugging info. In the admin browser, open a new tab and go to Site administration / Plugins / Caching / Configuration . Find the coursecattree cache area, and click Purge next to it. Wait for this to finish. In the user browser, reload the page. In the admin browser, reload the Manage courses and categories tab. EXPECTED: The user browser takes 10 seconds then displays the page, including debugging info about rebuilding the course cache. EXPECTED: The admin browser finishes immediately after the user browser does, and there should be no debugging info.

      Similar to MDL-61305, on a system with a large number of course categories, rebuilding the course category tree and writing it to cache can take a signifiant amount of time. This is especially true if the cache is stored on the file system, since the cache is written using set_many which creates a large number of small files (a separate issue which might be worth looking into).

      Since this cache is accessed on every page that displays the navigation, on a busy system is it currently possible that multiple requests will be trying to rebuild the cache at once, and thus re-writing to the same large number of small files at once. We have found this to cause severe CPU spikes on our file server, leading to system-wide performance issues.

      Ideally we should lock the rebuilding just like we have with the modinfo cache.

        1. after-fix.png
          after-fix.png
          49 kB
        2. before-fix.png
          before-fix.png
          39 kB
        3. image-2020-01-15-13-35-29-871.png
          image-2020-01-15-13-35-29-871.png
          274 kB
        4. screenshot-1.png
          screenshot-1.png
          38 kB

            marxjohnson Mark Johnson
            marxjohnson Mark Johnson
            Tim Hunt Tim Hunt
            Eloy Lafuente (stronk7) Eloy Lafuente (stronk7)
            Janelle Barcega Janelle Barcega
            Votes:
            1 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated:
              Resolved:

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0 minutes
                0m
                Logged:
                Time Spent - 1 hour, 20 minutes
                1h 20m

                  Error rendering 'clockify-timesheets-time-tracking-reports:timer-sidebar'. Please contact your Jira administrators.