Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-37761

Improve backup/restore within Moodle (e.g. course and activity duplication)

XMLWordPrintable

    • MOODLE_24_STABLE
    • MOODLE_26_STABLE
    • Hide

      Note: For easiest testing, start with a clean install of Moodle (highly recommended).
      This is because we need to test restoration of de-duplicated files and want to 100% ensure that the files aren't already in use elsewhere in the interface.

      These testing instructions try to test:

      • file-less course import
      • file-less activity duplication
      • that there are no backup regressions
      • that the file existence checks are suitable, specifically:
        • changes to restore from a contenthash work
        • that we don't blow up when a file no longer exists and we can't find it anywhere
      1. Create a new course
      2. Back it up as is (download the backup file)
        • Confirm that the backup log states that the file inclusion setting has been set to '1'
        • Confirm that the backup completed as expected
      3. Turn editing on and add:
        • an image as a label
        • a folder with some files
      4. Back up the course again (download the backup file)
        • Confirm that the backup log states that the file inclusion setting has been set to '1'
      5. Unzip the backup
        • Confirm the files are present in the files directory
        • Confirm that the files.xml file contains appropriate metadata
        • Confirm that moodle_backup.xml incudes a include_files setting of 1
      6. Duplicate the image label
        • Confirm that the backup log states that the file inclusion setting has been set to '0'
        • Confirm that the duplicate label appears correctly
      7. Duplicate the folder
        • Confirm that the backup log states that the file inclusion setting has been set to '0'
        • Confirm that the duplicate folder appears correctly with all files present
      8. Edit backup/backup.php to change the mode from MODE_GENERAL to MODE_SAMESITE
      9. Back up the course again (download the backup file)
        • Confirm that the backup log states that the file inclusion setting has been set to '0'
      10. Unzip the backup
        • Confirm the files are NOT present in the files directory
        • Confirm that the files.xml file contains appropriate metadata
        • Confirm that moodle_backup.xml incudes a include_files setting of 0
      11. Delete the course in it's entirety
      12. Run the attached script (37761-trashpurge.php)
        • WARNING: This removes your trash directory immediately instead of after 4 days
        • Confirm that the files are now gone from the files table (if you started with a fresh site, the table should be mostly empty)
      13. Create a new course
      14. Restore from the most recent backup (the one generated by a hacked backup.php)
        • Confirm that you were shown a warning about missing files
      15. View the course again
        • Confirm that the files are missing
      16. Put all of the files into the course again (just drag/drop them into the course somewhere)
      17. Delete the course again
      18. Create a new course
      19. Restore from the most recent backup (the one generated by a hacked backup.php)
        • Confirm that you were NOT shown a warning about missing files
      20. View the course again
        • Confirm that the files are all present now (they should be restored based upon the contenthash)
      21. Create a new course
      22. Use the Import option to import from the other course
        • Confirm that the backup log states that the file inclusion setting has been set to '0'
      23. View the new course
        • Confirm that all of the files were displayed correctly
      24. Make a backup from an older version of Moodle and a course including some files
      25. Create a new course
      26. Restore from the older backup
        • Confirm that you were NOT shown a warning about missing files
        • Confirm that you were NOT shown any errors (at least none you don't expect)
        • Confirm that the files imported correctly from the backup
      Show
      Note: For easiest testing, start with a clean install of Moodle (highly recommended). This is because we need to test restoration of de-duplicated files and want to 100% ensure that the files aren't already in use elsewhere in the interface. These testing instructions try to test: file-less course import file-less activity duplication that there are no backup regressions that the file existence checks are suitable, specifically: changes to restore from a contenthash work that we don't blow up when a file no longer exists and we can't find it anywhere Create a new course Back it up as is (download the backup file) Confirm that the backup log states that the file inclusion setting has been set to '1' Confirm that the backup completed as expected Turn editing on and add: an image as a label a folder with some files Back up the course again (download the backup file) Confirm that the backup log states that the file inclusion setting has been set to '1' Unzip the backup Confirm the files are present in the files directory Confirm that the files.xml file contains appropriate metadata Confirm that moodle_backup.xml incudes a include_files setting of 1 Duplicate the image label Confirm that the backup log states that the file inclusion setting has been set to ' 0 ' Confirm that the duplicate label appears correctly Duplicate the folder Confirm that the backup log states that the file inclusion setting has been set to ' 0 ' Confirm that the duplicate folder appears correctly with all files present Edit backup/backup.php to change the mode from MODE_GENERAL to MODE_SAMESITE Back up the course again (download the backup file) Confirm that the backup log states that the file inclusion setting has been set to ' 0 ' Unzip the backup Confirm the files are NOT present in the files directory Confirm that the files.xml file contains appropriate metadata Confirm that moodle_backup.xml incudes a include_files setting of 0 Delete the course in it's entirety Run the attached script (37761-trashpurge.php) WARNING: This removes your trash directory immediately instead of after 4 days Confirm that the files are now gone from the files table (if you started with a fresh site, the table should be mostly empty) Create a new course Restore from the most recent backup (the one generated by a hacked backup.php) Confirm that you were shown a warning about missing files View the course again Confirm that the files are missing Put all of the files into the course again (just drag/drop them into the course somewhere) Delete the course again Create a new course Restore from the most recent backup (the one generated by a hacked backup.php) Confirm that you were NOT shown a warning about missing files View the course again Confirm that the files are all present now (they should be restored based upon the contenthash) Create a new course Use the Import option to import from the other course Confirm that the backup log states that the file inclusion setting has been set to ' 0 ' View the new course Confirm that all of the files were displayed correctly Make a backup from an older version of Moodle and a course including some files Create a new course Restore from the older backup Confirm that you were NOT shown a warning about missing files Confirm that you were NOT shown any errors (at least none you don't expect) Confirm that the files imported correctly from the backup

      When copying a course (e.g. create + import), and possibly when duplicating an activity, the backup system currently backs up all files to temp and then restores that backup.

      Given we have data deduplication handled by the database, there's no need to actually back the files up and restore them - this just leads to increased disk utilisation for the backup process and the restore process (and possibly additional CPU time to recalculate the checksums).

      We should add a flag to the backup system to indicate when it should not backup files, and when it should not zip backups.

        1. 37761-trashpurge.php
          2 kB
          Andrew Lyons

            dobedobedoh Andrew Lyons
            dobedobedoh Andrew Lyons
            Sam Marshall Sam Marshall
            Dan Poltawski Dan Poltawski
            David Monllaó David Monllaó
            Votes:
            5 Vote for this issue
            Watchers:
            18 Start watching this issue

              Created:
              Updated:
              Resolved:

                Error rendering 'clockify-timesheets-time-tracking-reports:timer-sidebar'. Please contact your Jira administrators.