Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-48166

Major performance issue when editing course categories or courses

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Duplicate
    • Affects Version/s: 2.6.4, 3.3.2, 3.5.2
    • Fix Version/s: None
    • Component/s: Caching, Performance
    • Labels:
    • Affected Branches:
      MOODLE_26_STABLE, MOODLE_33_STABLE, MOODLE_35_STABLE

      Description

      Analysis

      Environment

      We are using Moodle 2.6.4+ containing about 3200 course categories in one single instance and Moodle File cache.
      For profiling reasons we've installed XHProf.

      Problem Reproduction

      1. login as user with course categories editing rights
      2. go to a editable course category
      3. click on "Edit this category"
      4. do any changes and click on "Save changes"

      What we know

      • There are several caching frameworks
        • File cache
        • memcache (We can't use this in our environment)
        • memcaches (We also can't use this in our environment)
        • MongoDB
      • There are several caching scopes
        • application
        • session
        • static
      • There is a course categories tree cache (in the application cache)
      • The coursecattree is needed for building the navigation e.g. on the "Course and category management" page
      • There are several invalidation-events that trigger the purging of the cache
      • There is an event called "changesincoursecat" that is purging the whole application cache

      Problem Analysis

      When editing a course category the coursecattree-cache will be purged and the user is forwarded to the "Course and category management" page, where parts of the coursecattree-cache are needed. Because of the reason that the coursecattree-cache is invalid at this moment the tree has to be rebuilt. On a local environment this step takes about 15 seconds. At this step about 6000 cache-files are written (2.5 ms per file).
      The coursecattree-cache is an application cache. During the regeneration of this application cache, the cache is locked for other users until the cache is generated. This leads to long waiting times. The productive environment shows waiting times up to 2 minutes.
      We checked the problem with a small Moodle installation and counldn't reproduce the problem. Therefore the problem must be in coincidence with the large number of course categories.
      We also tested with Moodle 2.6.5+ and 2.7.2+. Same problems.

      Profiling and Call Graphs

      As you can see the Call Graph with purged Cache shows a total time of 25838.202 ms where writing the files consumes 12607.197 ms (= 5940 files (= 2 x 2970 course categories)).

      Related Issues

      We already had a look at some related Moodle issues. No success...


      Hopefully someone can help.

        Attachments

        1. 2014-11-05_withCaching.mpr
          65 kB
        2. 2014-11-05_withPurgedCache.mpr
          69 kB
        3. moodle.png
          moodle.png
          30 kB
        4. withCaching.png
          withCaching.png
          2.41 MB
        5. withPurgedCache.png
          withPurgedCache.png
          638 kB

          Issue Links

            Activity

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              Diablo Manuel Gogl
              Participants:
              Component watchers:
              Matteo Scaramuccia, Amaia Anabitarte, Carlos Escobedo, Ferran Recio, Ilya Tregubov, Sara Arjona (@sarjona), Matteo Scaramuccia, Andrew Lyons, Dongsheng Cai, Huong Nguyen, Jun Pataleta, Michael Hawkins, Shamim Rezaie, Simey Lameze
              Votes:
              15 Vote for this issue
              Watchers:
              20 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: