Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-53849

Configurable refresh time for filesystem repository file synchronization

    XMLWordPrintable

    Details

    • Testing Instructions:
      Hide
      • Go to Site administration->Plugins->Repositories->File system,
        • (if you have no File system repository, you need to enable it via "Manage Repositories")
      • Look at "Repositories instances of the site": if there is a repository, click on "settings",
        • If there is none, click "Create a repository instance" and setup a new repository (there must be a directory in moodledata/repository so it will show up in a combo box)
      • Input a low Update interval value eg. 3 (for 3 seconds),
      • Save,
      • If you have no files in the directory your repository is pointing to, create some,
      • Open a course, turn editing on,
      • Click on "Add an activity or resource",
      • From "Resources", choose "File",
      • Input Name and Description
      • In "Select files" click on "Add" icon
      • On the left, there will be a repository you have just created/edited, select it,
      • Choose a file from this repository,
      • Check "Create an alias/shortcut to the file" and click "Select this file",
      • Click "Save and return to course",
      • Open database tool (ie. phpmyadmin), in table files_reference look at the latest row. There is a value in lastsync column - write it down,
      • In your course, click "Edit" and then "Edit settings" next to the added resource,
      • Click "Save and return to course",
      • Now look again at the same row in db - lastsync value should be higher, write it down,
      • Go back to Site administration->Plugins->Repositories->File system,
      • Click settings for your repository, input a large value for Update interval (eg. 86400 for 24hrs),
      • Save,
      • Go to your course and edit resource again,
      • Look once again at the lastsync value - it should not change
      Show
      Go to Site administration->Plugins->Repositories->File system, (if you have no File system repository, you need to enable it via "Manage Repositories") Look at "Repositories instances of the site": if there is a repository, click on "settings", If there is none, click "Create a repository instance" and setup a new repository (there must be a directory in moodledata/repository so it will show up in a combo box) Input a low Update interval value eg. 3 (for 3 seconds), Save, If you have no files in the directory your repository is pointing to, create some, Open a course, turn editing on, Click on "Add an activity or resource", From "Resources", choose "File", Input Name and Description In "Select files" click on "Add" icon On the left, there will be a repository you have just created/edited, select it, Choose a file from this repository, Check "Create an alias/shortcut to the file" and click "Select this file", Click "Save and return to course", Open database tool (ie. phpmyadmin), in table files_reference look at the latest row. There is a value in lastsync column - write it down, In your course, click "Edit" and then "Edit settings" next to the added resource, Click "Save and return to course", Now look again at the same row in db - lastsync value should be higher, write it down, Go back to Site administration->Plugins->Repositories->File system, Click settings for your repository, input a large value for Update interval (eg. 86400 for 24hrs), Save, Go to your course and edit resource again, Look once again at the lastsync value - it should not change
    • Affected Branches:
      MOODLE_30_STABLE
    • Pull from Repository:
    • Pull Master Branch:
      MDL-53849_configurable_refresh_time_for_filesystem_repository_file_synchronization

      Description

      At the moment class repository_filesystem, method sync_reference has code:

      if ($file->get_referencelastsync() + 60 > time()) {
          // Does not cost us much to synchronise within our own filesystem, check every 1 minute.
          return false;
      }
      

      When file metadata is being synchronized for a course with a lot of files, access to the course is practically blocked and DB usage is huge. This affects big courses (with a lot of resources) when course cache is built.
      Moodle has a hardcoded setting to synchronize a file if it has been synchronized more than 1 minute ago. This does not take into account that:

      • There may be a lot of files to synchronize
      • The access to the filesystem may be slow (e.g. over NFS)

      I suggest that we replace "60" hardcoded there with a configurable option, where administrator can choose how "fresh" the file's metadata should be. In many cases (like in mine), the files from external repository change very infrequently and this 60 seconds check creates huge performance bottleneck.

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated: