Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-69892

Moving many courses very quickly can make Moodle unusable on busy sites

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 3.8.5
    • Fix Version/s: None
    • Component/s: Caching, Performance
    • Labels:
      None
    • Affected Branches:
      MOODLE_38_STABLE

      Description

      We have a course archiving process that moves course that have finished running to an archive category (which restricts the ability of users to make changes to the courses) at the end of an academic year. This year it seems to have caused Moodle to become unusable while it was moving the courses, in previous years we have not noticed any slowdowns during the process.

      The scheduled task doing this was moving the courses in batches of 2500 every even hour, each batch was taking about 20 minutes.

      Looking at some of the slow traces we could see that there was a file lock in affect for over two minutes in most of them while the page was trying to get the category tree.

      My conclusion is that it is caused by the changes in MDL-67674, because we are moving so many courses so quickly I think that the cache entries are constantly being invalided and since only one user can now rebuild the cache entry it meant that most users where constantly waiting for the lock so that they could have their go at generating the entry that would then be almost immediately invalid again.

      The slowdowns lined up with when the task was scheduled to run:

      Memecache writes were rather high:

      A typical slow trace:

      My guess is that in pervious years we would have had more activity on our database when doing this as users would have been able to regenerate cache entries in parallel (and possibly even more writing to our cache server; but end users were less affected by that.

        Attachments

        1. periods-of-slowdown.png
          periods-of-slowdown.png
          53 kB
        2. memcache-profile.png
          memcache-profile.png
          42 kB
        3. slow-trace-summary.png
          slow-trace-summary.png
          64 kB
        4. slowtrace-details.png
          slowtrace-details.png
          97 kB

          Issue Links

            Activity

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              nmagill Neill Magill
              Participants:
              Component watchers:
              Matteo Scaramuccia, Amaia Anabitarte, Carlos Escobedo, Ferran Recio, Ilya Tregubov, Sara Arjona (@sarjona), Matteo Scaramuccia, Andrew Nicols, Dongsheng Cai, Huong Nguyen, Jun Pataleta, Michael Hawkins, Shamim Rezaie, Simey Lameze
              Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated: