Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-67674

Performance: Course category tree cache can get built in parallel

    XMLWordPrintable

    Details

    • Testing Instructions:
      Hide

      Note, this requires a small code change to simulate a slow rebuild of the coursecattree cache.

      • As admin, log in to your test site and go to the Site administration / Courses / Manage courses and categories
      • Open a second browser, log in as a second user account with the moodle/core:viewcourselist capability, and go to the course list (course/index.php)
      • Make a code change (just for testing) to course/classes/category.php:
        1. Search for the comment: // Re-build the tree.
        2. Immediately above this line, add the following line: sleep(10); debugging('Actually building course category tree');
      • Using the admin browser, click the up or down arrow on a category to change the category order (changing the order will invalidate the cache). Once this as completed, reload the page.
      • Using the other (user) browser, reload the current page. (You need to do this step within ten seconds - ideally a couple of seconds after reloading the admin browser.)
        • EXPECTED: The admin browser takes 10 seconds then displays debugging info about rebuilding the course category tree.
        • EXPECTED: The other browser finishes slightly faster - it should load immediately after the admin browser does. There should be no debugging info.
      • In the admin browser, open a new tab and go to Site administration / Plugins / Caching / Configuration.
      • Find the coursecattree cache area, and click Purge next to it.
      • Wait for this to finish.
      • In the user browser, reload the page.
      • In the admin browser, reload the Manage courses and categories tab.
        • EXPECTED: The user browser takes 10 seconds then displays the page, including debugging info about rebuilding the course cache.
        • EXPECTED: The admin browser finishes immediately after the user browser does, and there should be no debugging info.
      Show
      Note, this requires a small code change to simulate a slow rebuild of the coursecattree cache. As admin, log in to your test site and go to the Site administration / Courses / Manage courses and categories Open a second browser, log in as a second user account with the moodle/core:viewcourselist capability, and go to the course list ( course/index.php ) Make a code change (just for testing) to course/classes/category.php: Search for the comment:  // Re-build the tree. Immediately above this line, add the following line: sleep(10); debugging('Actually building course category tree'); Using the admin browser, click the up or down arrow on a category to change the category order (changing the order will invalidate the cache). Once this as completed, reload the page. Using the other (user) browser, reload the current page. (You need to do this step within ten seconds - ideally a couple of seconds after reloading the admin browser.) EXPECTED: The admin browser takes 10 seconds then displays debugging info about rebuilding the course category tree. EXPECTED: The other browser finishes slightly faster - it should load immediately after the admin browser does. There should be no debugging info. In the admin browser, open a new tab and go to Site administration / Plugins / Caching / Configuration . Find the coursecattree cache area, and click Purge next to it. Wait for this to finish. In the user browser, reload the page. In the admin browser, reload the Manage courses and categories tab. EXPECTED: The user browser takes 10 seconds then displays the page, including debugging info about rebuilding the course cache. EXPECTED: The admin browser finishes immediately after the user browser does, and there should be no debugging info.
    • Affected Branches:
      MOODLE_37_STABLE, MOODLE_38_STABLE, MOODLE_39_STABLE
    • Fixed Branches:
      MOODLE_37_STABLE, MOODLE_38_STABLE
    • Pull 3.7 Branch:
      MDL-67674_37_STABLE
    • Pull 3.8 Branch:
      MDL-67674_38_STABLE
    • Pull Master Branch:
      MDL-67674_master

      Description

      Similar to MDL-61305, on a system with a large number of course categories, rebuilding the course category tree and writing it to cache can take a signifiant amount of time. This is especially true if the cache is stored on the file system, since the cache is written using set_many which creates a large number of small files (a separate issue which might be worth looking into).

      Since this cache is accessed on every page that displays the navigation, on a busy system is it currently possible that multiple requests will be trying to rebuild the cache at once, and thus re-writing to the same large number of small files at once. We have found this to cause severe CPU spikes on our file server, leading to system-wide performance issues.

      Ideally we should lock the rebuilding just like we have with the modinfo cache.

        Attachments

        1. after-fix.png
          after-fix.png
          49 kB
        2. before-fix.png
          before-fix.png
          39 kB
        3. image-2020-01-15-13-35-29-871.png
          image-2020-01-15-13-35-29-871.png
          274 kB
        4. screenshot-1.png
          screenshot-1.png
          38 kB

          Issue Links

            Activity

              People

              Assignee:
              marxjohnson Mark Johnson
              Reporter:
              marxjohnson Mark Johnson
              Peer reviewer:
              Tim Hunt
              Integrator:
              Eloy Lafuente (stronk7)
              Tester:
              Janelle Barcega
              Participants:
              Component watchers:
              Matteo Scaramuccia, Amaia Anabitarte, Carlos Escobedo, Ferran Recio, Sara Arjona (@sarjona), Víctor Déniz Falcón
              Votes:
              1 Vote for this issue
              Watchers:
              7 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:
                Fix Release Date:
                9/Mar/20

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 1 hour, 20 minutes
                  1h 20m