-
Improvement
-
Resolution: Fixed
-
Minor
-
4.0
-
MOODLE_400_STABLE
-
MOODLE_400_STABLE
-
MDL-72328-master -
Redis caches do not have TTL support. Even if TTL is set for a cache (e.g. core/completion) then no items in that cache will ever be deleted.
To take the example of core/completion (which is the one that showed up as a problem in our automated load test, although it is slightly less of a problem in reality):
- There is one item in this cache for each user who visits a course with completion on (i.e. the number of items goes up to {users} * {courses that they visit}).
- For a fairly large course with approx. 500 activities, that item will be about 80 KB, call it 100 KB including Redis overhead.
- Supposing 10,000 different users visit the site each day, and only visit one course each, this would occupy approximately 100 * 10,000 KB = 1 GB of Redis cache per day.
- Eventually this will stop increasing at that rate because some of tomorrow's users will be the same people as today, but it's still wasting a lot of memory on Redis for something that's only valid for an hour.
The reason this is less of a problem in reality is that stupid stuff means that while staff are editing websites (which wasn't included in our load test), core/completion cache gets purged sometimes. Still, at least conceptually, it is a bit rubbish.
Incidentally, Redis features such as automatically deleting keys when it runs low on memory would mitigate this problem, but they can't be used entirely satisfactorily on a Moodle setup because it can only delete hashes (i.e. purge entire caches) and not delete single items within a hash (individual keys).
It would be relatively easy to implement TTL support for Redis caches within Moodle by using the Redis 'sorted list' data structure to hold expiry dates, and then a scheduled task to delete old items based on that structure. I have an implementation which I will contribute.
This means that cache set and delete are 2x slower, only for caches using TTL, but they were very fast anyway and generally, get performance (which is unaffected) is much more significant.