Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-65729

Ad-hoc task processing for automated backups

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Waiting for peer review
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 3.7.1
    • Fix Version/s: None
    • Component/s: Backup
    • Testing Instructions:
      Hide
      1. Create a few courses
      2. Purge caches
      3. Go to Administration > Site administration > Courses > Backups > Automated backup setup
      4. Set backup_auto_active to enabled
      5. Choose all days of week
      6. run ```php admin/tool/task/cli/schedule_task.php --execute=
        core
        task
        automated_backup_task```
      7.  Find out what course has been backed up from the CLI. We could create new course to keep course backup pushed to adhoc task instead of hacking the code.
      8. Go to the course which there is a back up job pushed to adhoc task queue. Go to restore section. There should not any back up ready there. Before this change, there should be a backup there already. Or go to backup report, should see some course backup are queued
      9. run ```php admin/cli/cron.php``` to execute the adhoc tasks. 
      10. Now go back to the course, there should be a back up in the Restore section
      Show
      Create a few courses Purge caches Go to  Administration  >  Site administration  >  Courses  >  Backups  >  Automated backup setup Set  backup_auto_active  to  enabled Choose all days of week run ```php admin/tool/task/cli/schedule_task.php --execute= core task automated_backup_task```  Find out what course has been backed up from the CLI. We could create new course to keep course backup pushed to adhoc task instead of hacking the code. Go to the course which there is a back up job pushed to adhoc task queue. Go to restore section. There should not any back up ready there. Before this change, there should be a backup there already. Or go to backup report, should see some course backup are queued run ```php admin/cli/cron.php``` to execute the adhoc tasks.  Now go back to the course, there should be a back up in the Restore section
    • Affected Branches:
      MOODLE_37_STABLE
    • Pull from Repository:
    • Pull Master Branch:
      master-MDL-65729

      Description

      Currently the processing of automated backups runs a single long running cron task with its own bespoke locking.
      One problem here is in a Moodle site that has a lot of courses that change often, the rate of change of the courses can be to much for the current way of processing automated backups. This means that automated course backups "get behind" and backups don't happen in a timely manner.
      MDL-65201 is a first step to making this better by improving how courses are ordered for automated backups.
      This request is to further refactor the automated backup code to:

      • Split individual backups into ad-hoc tasks. This gives the ability to run several automated backups in parallel and increase the overall throughput of the processing.
      • Remove the custom locking that is in the automated backup code and make automated backups use the regular scheduled task locking.

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                2 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Time Tracking

                  Estimated:
                  Original Estimate - 0 minutes
                  0m
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 1 day, 2 hours, 35 minutes
                  1d 2h 35m