Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-35773

Include files option in backup settings

XMLWordPrintable

    • Any
    • MOODLE_23_STABLE, MOODLE_31_STABLE, MOODLE_33_STABLE, MOODLE_34_STABLE, MOODLE_35_STABLE, MOODLE_36_STABLE, MOODLE_38_STABLE
    • MOODLE_38_STABLE
    • MDL-35773_master
    • Hide
      Initial setup
      1. Visit 'Site administration' > 'Development' > 'Make test course'.
      2. Create a small course.
      3. Visit the course.
      4. Backup the course with default values.
      5. Visit the restore area for this course.
      6. Check the size of the file in the backup area (should be approx ~9mb).
      Fileless UI backup
      1. Visit the course you created in the initial step.
      2. Backup the course via the UI but make sure to de-select “Include files” in the backup options.
      3. Check the backup completed successfully.
      4. Check the size of the backup and confirm it is much smaller than the initial backup file.
      5. Restore this backup into a new course.
      6. Check backup restore successfully and all files can be accessed in new course.
      Fileless Automated backup
      1. Visit 'Site administration' > 'Courses' > 'Backups' > 'Automated backup setup'.
      2. Set 'Active' to 'Manual'.
      3. Uncheck 'Include files' and save.
      4. Run php admin/cli/automated_backups.php.
      5. Visit the course you created in the initial step.
      6. Visit the restore area for this course.
      7. Scroll down to 'Automated backups'.
      8. Confirm the file size is smaller than the initial backup file.
      9. Restore this backup into a new course.
      10. Check backup restore successfully and all files can be accessed in new course.
      Adjustable trash cleanup
      1. Visit the course you created in the initial step.
      2. Visit the restore area for this course.
      3. Download a fileless backup of the initial course (one of the smaller sized backups).
      4. Delete all copies of the initial course (including restores made in previous steps).
      5. Visit 'Site administration' > 'Courses' > 'Restore course'.
      6. Use backup you downloaded earlier and restore as a new course.
      7. Check that all files exist in restored course.
      Adjustable trash cleanup fail
      1. Visit 'Site administration' > 'Server' > 'Cleanup'.
      2. Set 'Clean trash pool files' to 1 second.
      3. Wait a couple of seconds.
      4. Run php admin/tool/task/cli/schedule_task.php --execute='\core\task\file_trash_cleanup_task'.
      5. Visit the mdl_files table and delete any records that have the filearea set to draft.
      6. Visit your moodledata directory and delete the contents of the trashdir if it exists.
      7. Visit 'Site administration' > 'Courses' > 'Restore course'.
      8. Use backup you downloaded earlier and restore as a new course.
      9. Course should restore but will be missing files.
      Recycle bin
      1. Log in as the admin.
      2. Visit 'Site administration' > 'Courses' > 'Backups' > 'Automated backup setup'.
      3. Uncheck 'Include files'.
      4. Click 'Save changes'.
      5. Create a course.
      6. Enrol a user as a student.
      7. Create an assignment.
      8. Log in as the student.
      9. Visit the course.
      10. Add a submission with a file for the assignment.
      11. Log in as the admin.
      12. Visit the course.
      13. Delete the assignment.
      14. Run php admin/cli/cron.php
      15. Visit the mdl_files table and delete any records that have the filearea set to draft.
      16. Go to your moodledata directory and delete the trashdir if it exists.
      17. Visit the recycle bin by clicking on the course settings cog on the top-right (if you are using Boost).
      18. Click to restore the assignment.
      19. Visit the assignment.
      20. Confirm the file is still present.
      Show
      Initial setup Visit 'Site administration' > 'Development' > 'Make test course'. Create a small course. Visit the course. Backup the course with default values. Visit the restore area for this course. Check the size of the file in the backup area (should be approx ~9mb). Fileless UI backup Visit the course you created in the initial step. Backup the course via the UI but make sure to de-select “Include files” in the backup options. Check the backup completed successfully. Check the size of the backup and confirm it is much smaller than the initial backup file. Restore this backup into a new course. Check backup restore successfully and all files can be accessed in new course. Fileless Automated backup Visit 'Site administration' > 'Courses' > 'Backups' > 'Automated backup setup'. Set 'Active' to 'Manual'. Uncheck 'Include files' and save. Run php admin/cli/automated_backups.php . Visit the course you created in the initial step. Visit the restore area for this course. Scroll down to 'Automated backups'. Confirm the file size is smaller than the initial backup file. Restore this backup into a new course. Check backup restore successfully and all files can be accessed in new course. Adjustable trash cleanup Visit the course you created in the initial step. Visit the restore area for this course. Download a fileless backup of the initial course (one of the smaller sized backups). Delete all copies of the initial course (including restores made in previous steps). Visit 'Site administration' > 'Courses' > 'Restore course'. Use backup you downloaded earlier and restore as a new course. Check that all files exist in restored course. Adjustable trash cleanup fail Visit 'Site administration' > 'Server' > 'Cleanup'. Set 'Clean trash pool files' to 1 second. Wait a couple of seconds. Run php admin/tool/task/cli/schedule_task.php --execute='\core\task\file_trash_cleanup_task' . Visit the mdl_files table and delete any records that have the filearea set to draft . Visit your moodledata directory and delete the contents of the trashdir if it exists. Visit 'Site administration' > 'Courses' > 'Restore course'. Use backup you downloaded earlier and restore as a new course. Course should restore but will be missing files. Recycle bin Log in as the admin. Visit 'Site administration' > 'Courses' > 'Backups' > 'Automated backup setup'. Uncheck 'Include files'. Click 'Save changes'. Create a course. Enrol a user as a student. Create an assignment. Log in as the student. Visit the course. Add a submission with a file for the assignment. Log in as the admin. Visit the course. Delete the assignment. Run php admin/cli/cron.php Visit the mdl_files table and delete any records that have the filearea set to draft . Go to your moodledata directory and delete the trashdir if it exists. Visit the recycle bin by clicking on the course settings cog on the top-right (if you are using Boost). Click to restore the assignment. Visit the assignment. Confirm the file is still present.

      When writing scripts which backup and then immediately restore to the same site, it would greatly improve performance if the system did not copy the files.

      In real usage it is easily possible for a course to be multiple gigabytes in size, and networked file systems are not necessarily all that fast, so it can add a minute or more to the time taken, potentially increasing the likelihood of timeouts. (Some front end load balancing webservers that direct requests time out after a few minutes if there is no output.)

      Using the import mode it is possible to prevent it from zipping the overall backup, but it is not currently possible (I think) to stop it from saving files. Also the restore does not work if the files are not present, even though it knows the content hash and therefore can check the file already exists.

      As well as custom external scripts, this would potentially also improve performance for the Moodle core 'duplicate' feature as I presume this also copies the file in the same manner. This might improve user experience a bit in that case, although with a single file it's obviously unlikely to be so huge it would actually time out.

      Eloy suggests the following setting:

      ->USE_AT_YOUR_OWN_RISK_im_gonna_restore_immediately_into_samesite_guy=true;

      Note we do not need this urgently or anything, it is just a nice-to-have that came up during development here.

        1. MDL-35773.jpg
          MDL-35773.jpg
          59 kB
        2. MDL-35773 (2).jpg
          MDL-35773 (2).jpg
          46 kB
        3. MDL-35773 (3).jpg
          MDL-35773 (3).jpg
          24 kB
        4. MDL-35773 (4).jpg
          MDL-35773 (4).jpg
          55 kB
        5. Screenshot from 2019-07-10 20-10-32.png
          Screenshot from 2019-07-10 20-10-32.png
          74 kB

            markn Mark Nelson
            quen Sam Marshall
            Dmitrii Metelkin Dmitrii Metelkin
            Jake Dallimore Jake Dallimore
            Anna Carissa Sadia Anna Carissa Sadia
            Votes:
            5 Vote for this issue
            Watchers:
            17 Start watching this issue

              Created:
              Updated:
              Resolved:

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0 minutes
                0m
                Logged:
                Time Spent - 2 days, 30 minutes
                2d 30m

                  Error rendering 'clockify-timesheets-time-tracking-reports:timer-sidebar'. Please contact your Jira administrators.