-
Improvement
-
Resolution: Fixed
-
Minor
-
4.0.4, 4.0.5
-
MOODLE_400_STABLE
-
MOODLE_402_STABLE
-
MDL-76129-master -
The update_capabilities function in accesslib is called when installing a new plugin or updating an existing one. Performance seems quite poor. (I detected this as a result of install logging added in MDL-75751.)
It is a bit slower than it ought to be in standard Moodle, but in OU Moodle we noticed it takes almost 17 minutes to run this function when installing mod_bigbluebuttonbn, which seems quite ridiculous (it doesn't even have a particularly large number of capabilities).
In addition, the admin_apply_default_settings function (it's not timed in MDL-75751, but I noticed from getting annoyed having to wait for it at the end of each run when testing other changes) also seems slower than necessary.
After investigation, I discovered there are three reasons for this, which are all addressed in separate commits within this issue:
1) Adding new capabilities is inefficient when there is more than one capability because it clears the capability cache after adding each capability to the database, then rebuilds the cache each time. I have changed it to add all capabilities to the database and then clear it once only.
2) Adding new capabilities that clone the value of an existing capability can be slow if there are lots of places where the existing capability is set/overridden. It's possible to roughly halve the number of queries required for this if we extend a database query and add a way to tell assign_capability that it doesn't have to bother checking some details.
3) (Related to point 1) During install/upgrade, caching is disabled. This causes disastrously poor performance in update_capabilities and quite poor performance in admin_apply_default_settings, which both appear to heavily rely on caches. I made a way to temporarily enable in-memory caching.
Together, changes 1 and 3 improve performance of a fresh Moodle install (phpunit init) on my Windows developer laptop by about 60% (13 minutes -> 5 minutes).
I anticipate that these fixes will also significantly improve the performance of upgrades like the OU's Moodle 4 upgrade, removing that 17 minute delay (I can't test that scenario, but I can't see anything else wrong with it now).
Note: We are are not actually relying on this change for our Moodle 4 upgrade, because it might not be ready in time even if we backport. Instead, we are going to manually insert capabilities into the database just before the upgrade, which should avoid the long delay. However, having done the investigation, it seems like this could be a big advantage for core Moodle.
The third point above is probably most controversial. There are good reasons to disable caching during upgrade/install:
- Upgrade/install can work even if cache infrastructure has problems.
- There may be drastic database changes in areas that are normally stable, and using cached data could result in inconsisten behaviour.
- If we did in-memory caching throughout large parts of upgrade/install then it would probably occupy a lot of memory.
To deal with all these issues, my solution is to enable caching only very temporarily (during the run-time of one function) and discard caches afterwards. Caching is not enabled during any of the 'API' parts of upgrade/install, or the other parts that make database structure changes (upgrade.php/install.php/install.xml). I hope this is a safe solution.