Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.6
    • Fix Version/s: 2.6
    • Component/s: Caching
    • Labels:
    • Rank:
      51368

      Description

      The difference from normal $CFG->cachedir is that the directory does not have to be shared by all cluster nodes, the files stored in $CFG->localcachedir MUST NOT change! Default location is "$CFG->dataroot/localcache".

      All files there must use unique revision numbers or hashes because the cached data can not be invalidated by any means - it will support only adding of new files, but no file deletes or overrides. The $CFG->localcachedir will be growing over time, all local cache files will be deleted during purge_all_caches() (usually upgrade).

      When using hashes please make sure the number of files in one directory is kept to some reasonable number because some filesystems might have problems with large number of files in one directory.

      Design requirements:

      • Everything must be automatic without extra administrative overhead.
      • purge_all_caches() on any node triggers local cache purging on all other nodes before adding any new files there.
      • Performance on standalone servers must not be worse.

      Usage restrictions:

      • No file deletes
      • No file modifications
      • No file overwriting
      • Soft limit on maximum number of files in one directory (few thousands)

      Benefits:

      • clustered servers may use local tempfs
      • no opcache cache invalidation problems if php files included from localcachedir (because the files must not change)

      Converted areas:

      1. lib/javascript.php
      2. theme/*

        Issue Links

          Activity

          Hide
          Petr Škoda added a comment -

          Submitting for peer review...

          Show
          Petr Škoda added a comment - Submitting for peer review...
          Hide
          Peter Bulmer added a comment -

          As a general discussion point, I imagine that using unix timestamps for revisions would be easier, as it means that old revisions could easily be identified by scheduled webserver job and deleted (ie no need to query database to determine which is the current revision).

          Show
          Peter Bulmer added a comment - As a general discussion point, I imagine that using unix timestamps for revisions would be easier, as it means that old revisions could easily be identified by scheduled webserver job and deleted (ie no need to query database to determine which is the current revision).
          Hide
          Petr Škoda added a comment - - edited

          timestamps only? I am afraid that would work only if you defined some fixed lifetime which is not enough in most use cases, we would have to store the reset after timestamp somewhere anyway.

          Show
          Petr Škoda added a comment - - edited timestamps only? I am afraid that would work only if you defined some fixed lifetime which is not enough in most use cases, we would have to store the reset after timestamp somewhere anyway.
          Hide
          Peter Bulmer added a comment - - edited

          One of us seems to be at the wrong end of the stick.

          My intended point was thus:
          if you have a directory structure:
          /var/cache/moodlelocalcache/lang/da39a3ee5e6b4b0d3255bfef95601890afd80709/file1
          /var/cache/moodlelocalcache/lang/da39a3ee5e6b4b0d3255bfef95601890afd80709/file2
          /var/cache/moodlelocalcache/lang/da39a3ee5e6b4b0d3255bfef95601890afd80709/fileN
          /var/cache/moodlelocalcache/lang/86f7e437faa5a7fce15d1ddcb9eaeaea377667b8/file1
          /var/cache/moodlelocalcache/lang/86f7e437faa5a7fce15d1ddcb9eaeaea377667b8/fileB
          /var/cache/moodlelocalcache/lang/86f7e437faa5a7fce15d1ddcb9eaeaea377667b8/fileZ

          System cron can't see which directory is current (da39a3ee5e6b4b0d3255bfef95601890afd80709 or 86f7e437faa5a7fce15d1ddcb9eaeaea377667b8) based on their names. To delete the old cache, it will need to connect to the database, query what is current, and delete everything else. Alternatively, moodle's cron will need to do this, but all webservers will need to be running moodle cron - I suspect some people run moodle cron from only one webserver.

          If instead you had a file structure
          /var/cache/moodlelocalcache/lang/1370000000/file1
          /var/cache/moodlelocalcache/lang/1370000000/fileB
          /var/cache/moodlelocalcache/lang/1370000000/fileZ
          /var/cache/moodlelocalcache/lang/1373495668/file1
          /var/cache/moodlelocalcache/lang/1373495668/file2
          /var/cache/moodlelocalcache/lang/1373495668/fileN

          Then immediately system cron can see that cache has cleared more recently than 1370000000, and as such can delete /var/cache/moodlelocalcache/lang/1370000000

          I don't see how this makes fixed lifetime necessary.

          In the event that you need to clear the cache more than once in a particular second, then the timestamp could be incremented - it need not matter that the number is a second or two in the future, only that it always increases & directories with the lower number can be deleted.

          Hopefully this makes sense.

          Show
          Peter Bulmer added a comment - - edited One of us seems to be at the wrong end of the stick. My intended point was thus: if you have a directory structure: /var/cache/moodlelocalcache/lang/da39a3ee5e6b4b0d3255bfef95601890afd80709/file1 /var/cache/moodlelocalcache/lang/da39a3ee5e6b4b0d3255bfef95601890afd80709/file2 /var/cache/moodlelocalcache/lang/da39a3ee5e6b4b0d3255bfef95601890afd80709/fileN /var/cache/moodlelocalcache/lang/86f7e437faa5a7fce15d1ddcb9eaeaea377667b8/file1 /var/cache/moodlelocalcache/lang/86f7e437faa5a7fce15d1ddcb9eaeaea377667b8/fileB /var/cache/moodlelocalcache/lang/86f7e437faa5a7fce15d1ddcb9eaeaea377667b8/fileZ System cron can't see which directory is current (da39a3ee5e6b4b0d3255bfef95601890afd80709 or 86f7e437faa5a7fce15d1ddcb9eaeaea377667b8) based on their names. To delete the old cache, it will need to connect to the database, query what is current, and delete everything else. Alternatively, moodle's cron will need to do this, but all webservers will need to be running moodle cron - I suspect some people run moodle cron from only one webserver. If instead you had a file structure /var/cache/moodlelocalcache/lang/1370000000/file1 /var/cache/moodlelocalcache/lang/1370000000/fileB /var/cache/moodlelocalcache/lang/1370000000/fileZ /var/cache/moodlelocalcache/lang/1373495668/file1 /var/cache/moodlelocalcache/lang/1373495668/file2 /var/cache/moodlelocalcache/lang/1373495668/fileN Then immediately system cron can see that cache has cleared more recently than 1370000000, and as such can delete /var/cache/moodlelocalcache/lang/1370000000 I don't see how this makes fixed lifetime necessary. In the event that you need to clear the cache more than once in a particular second, then the timestamp could be incremented - it need not matter that the number is a second or two in the future, only that it always increases & directories with the lower number can be deleted. Hopefully this makes sense.
          Hide
          Petr Škoda added a comment -

          Did you look at my proposed make_localcache_directory()? It does the cleanup based on a timestamp of a special new file in localcachedir + new setting with time of last purge_all_caches(). It does not need any cron in my opinion. There is little overhead because we already have the setting in $CFG and we do other file IOs before writing new things to cache. I have also made sure concurrent deletes do not cause any problems.

          /var/cache/moodlelocalcache/lang/1370000000/file1 structure is similar to our revision counters in themes, they are also based on current time but they might go a bit into the future.

          Show
          Petr Škoda added a comment - Did you look at my proposed make_localcache_directory()? It does the cleanup based on a timestamp of a special new file in localcachedir + new setting with time of last purge_all_caches(). It does not need any cron in my opinion. There is little overhead because we already have the setting in $CFG and we do other file IOs before writing new things to cache. I have also made sure concurrent deletes do not cause any problems. /var/cache/moodlelocalcache/lang/1370000000/file1 structure is similar to our revision counters in themes, they are also based on current time but they might go a bit into the future.
          Hide
          Peter Bulmer added a comment -

          Apologies
          I was reading your bit about files: "must use unique revision numbers or hashes", and thought you were meaning directories. This was built on the assumption that you wouldn't often need to be re-prefixing names in the cache, and could instead just clear the cache/bump the directory prefix.

          Just how often are things being deleted/reprefixed in the cache?

          Show
          Peter Bulmer added a comment - Apologies I was reading your bit about files: "must use unique revision numbers or hashes", and thought you were meaning directories. This was built on the assumption that you wouldn't often need to be re-prefixing names in the cache, and could instead just clear the cache/bump the directory prefix. Just how often are things being deleted/reprefixed in the cache?
          Hide
          Petr Škoda added a comment -

          The things in this cache may change at any time, it depends on the revision numbers. In case of themes for example it is after any theme setting change that might affect the generated CSS. The full purging of caches is done during upgrade or when admin does purge all caches manually. Please note the revision numbers are also used for invalidation of browser caches on client computers.

          Show
          Petr Škoda added a comment - The things in this cache may change at any time, it depends on the revision numbers. In case of themes for example it is after any theme setting change that might affect the generated CSS. The full purging of caches is done during upgrade or when admin does purge all caches manually. Please note the revision numbers are also used for invalidation of browser caches on client computers.
          Hide
          Petr Škoda added a comment -

          no objections so far, submitting for integration, thanks

          Show
          Petr Škoda added a comment - no objections so far, submitting for integration, thanks
          Hide
          Dan Poltawski added a comment -

          Integrated to master, thanks Petr.

          Show
          Dan Poltawski added a comment - Integrated to master, thanks Petr.
          Hide
          Eloy Lafuente (stronk7) added a comment -

          ohoh, this has been integrated and includes MDL-23493 that is being analyzed right now... let's see how the history ends...

          Show
          Eloy Lafuente (stronk7) added a comment - ohoh, this has been integrated and includes MDL-23493 that is being analyzed right now... let's see how the history ends...
          Hide
          Dan Poltawski added a comment -

          grr.

          Show
          Dan Poltawski added a comment - grr.
          Hide
          David Monllaó added a comment -

          It passes, tested in master, all resources with HTTP 200; unit tests also passing.

          Show
          David Monllaó added a comment - It passes, tested in master, all resources with HTTP 200; unit tests also passing.
          Hide
          Damyon Wiese added a comment -

          Thanks again for another week of fixes, improvements and testing. These changes have been released to the world.

          Cheers, Damyon

          Show
          Damyon Wiese added a comment - Thanks again for another week of fixes, improvements and testing. These changes have been released to the world. Cheers, Damyon
          Hide
          Martin Dougiamas added a comment -

          Where are the docs for this? So admins know how to use it?

          Show
          Martin Dougiamas added a comment - Where are the docs for this? So admins know how to use it?
          Hide
          Martin Dougiamas added a comment -

          In short it's this:

          "In Moodle 2.6 onwards make sure you set $CFG->localcachedir to some local directory in config.php (for each node). This will speed up some of the disk caching that happens outside of MUC, such as themes, javascript, libraries etc."

          I added that here http://docs.moodle.org/25/en/Caching#Other_performance_advice_for_load-balanced_web_servers

          Show
          Martin Dougiamas added a comment - In short it's this: "In Moodle 2.6 onwards make sure you set $CFG->localcachedir to some local directory in config.php (for each node). This will speed up some of the disk caching that happens outside of MUC, such as themes, javascript, libraries etc." I added that here http://docs.moodle.org/25/en/Caching#Other_performance_advice_for_load-balanced_web_servers
          Hide
          Petr Škoda added a comment -

          This setting in documented in config-dist.php and the caching page linked by Martin and Documentation link above. Thanks everybody.

          Show
          Petr Škoda added a comment - This setting in documented in config-dist.php and the caching page linked by Martin and Documentation link above. Thanks everybody.

            People

            • Votes:
              3 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: