Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-64329

Add caching to backups to skip already processed files/areas

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Development in progress
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 3.1.15, 3.3.9, 3.4.6, 3.5.3, 3.6.1
    • Fix Version/s: None
    • Component/s: Backup, Performance
    • Labels:
      None
    • Testing Instructions:
      Hide
      1. Install mod_hvp
      2. Create a new course
      3. Add a H5P Interactive Content activity to the course
      4. In the editor under "Select content type" use the "Get" button to download and install 3+ different content types
      5. Select one of those content types and finish creating the activity
      6. Add another H5P Interactive Content activity to the same course (using any content type)
      7. While running profiling, backup the course
      8. Add the caching patch
      9. While running profiling backup the course again
      10. Compare profiling runs, the caching patch should lower the number of record_exists calls and speed up the backup time

      Alternatively steps 2 - 6 can be skipped by restoring the attached backup (created in 3.7dev) and using that course.

      Show
      Install mod_hvp Create a new course Add a H5P Interactive Content activity to the course In the editor under "Select content type" use the "Get" button to download and install 3+ different content types Select one of those content types and finish creating the activity Add another H5P Interactive Content activity to the same course (using any content type) While running profiling, backup the course Add the caching patch While running profiling backup the course again Compare profiling runs, the caching patch should lower the number of record_exists calls and speed up the backup time Alternatively steps 2 - 6 can be skipped by restoring the attached backup (created in 3.7dev) and using that course.
    • Affected Branches:
      MOODLE_31_STABLE, MOODLE_33_STABLE, MOODLE_34_STABLE, MOODLE_35_STABLE, MOODLE_36_STABLE
    • Pull from Repository:
    • Pull Master Branch:

      Description

      Currently the annotate_files function in backup/util/dbops/backup_structure_dbops.class.php can be called with the same file area multiple times. The processing of the same files is handled in the insert_backup_ids_record function where it does a record_exists on the backup_ids_temp table. This ends up doing a lot of unnecessary record_exists calls, in our case we have a client that has a large file area (due to the way the H5P mod stores its library files) so a lot of time and db calls could be saved. 

      The idea is to use cache to create a list of already processed file areas and skip them if we've already processed them.

        Attachments

          Activity

            People

            • Assignee:
              rhell4 Rossco Hellmans
              Reporter:
              rhell4 Rossco Hellmans
              Peer reviewer:
              Matt Porritt
              Participants:
              Component watchers:
              Adrian Greeve, Mihail Geshoski, Peter Dias, Matteo Scaramuccia, Jake Dallimore, Jun Pataleta, Ryan Wyllie
            • Votes:
              4 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated: