Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-65857

Optimize mdl_files entries

    XMLWordPrintable

    Details

    • Testing Instructions:
      Hide

      We should test as much file-related functionality as possible. Some ideas:

      • Test file_trash_cleanup_task scheduled task
      • Run all available unit tests
      • Run all available functional tests
      • Create / edit / delete a resource in a course with single / many files and single / many sub-directories.
      Show
      We should test as much file-related functionality as possible. Some ideas: Test file_trash_cleanup_task scheduled task Run all available unit tests Run all available functional tests Create / edit / delete a resource in a course with single / many files and single / many sub-directories.
    • Affected Branches:
      MOODLE_38_STABLE
    • Pull Master Branch:
      MDL-65857_optimize_mdl_files_entries

      Description

      Every file stored in Moodle has a . (dot) entry in mdl_files. See https://moodle.org/mod/forum/discuss.php?d=257008 and https://moodle.org/mod/forum/discuss.php?d=387581.

      Those entries do not bring any new information for top-level directory ("/") - each resource simply has one entry like that and the entry is always identical. It's content hash is: da39a3ee5e6b4b0d3255bfef95601890afd80709

      Statistics from one of our databases:

      • mdl_files table has about 90 million records.
      • Out of those, there are about 20 million "." rows.
        Since this information is redundant, the 20 million rows could possibly be removed from the database and replaced with a bit of PHP logic when handling the files.

      Bear in mind that the "." entries are still needed for sub-directories. The optimization will help most in the installations with a lot of single file resources.

      I have implemented the optimization and checked all the places I could think of where that could cause some side effects. All units tests have been either updated or related code fixed. But of course, since this is a core change, there may be some side effect I have missed.

      The patch makes Moodle to try to avoid using ./ entry but as a fallback, fake mock-up entry for ./ is returned.

      Of course this change may break plugins that use the mdl_files table directly - but from what I can see if files API is used, then the risk is low.
      However - this change should be documented in the Moodle files documentation and in the dev changelog.

      I would suggest that the code changes are now reviewed by another developer. If we decide to include them in Moodle core and there are no obvious risks here, then we need to handle one more thing - an upgrade during which all top level "." entries are deleted.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                tmuras Tomasz Muras
                Peer reviewer:
                Brendan Heywood
                Participants:
                Component watchers:
                Matteo Scaramuccia, Jake Dallimore, Jun Pataleta
              • Votes:
                3 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated: