Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-65729

Ad-hoc task processing for automated backups

    XMLWordPrintable

Details

    • MOODLE_37_STABLE, MOODLE_38_STABLE
    • MOODLE_39_STABLE
    • master-MDL-65729
    • Hide
      1. Create a few courses
      2. Purge caches
      3. Go to Administration > Site administration > Courses > Backups > Automated backup setup
      4. Set backup_auto_active to enabled
      5. Choose all days of the week
      6. run

        php admin/tool/task/cli/schedule_task.php --execute=\\core\\task\\automated_backup_task

        from the command line.

      7. Go to "Site administration > Reports > Backups" and check that the status is set to "Automated backup pending"
      8. run the following sql command on you database (requires direct access to the database)

        UPDATE "mdl_backup_courses" SET nextstarttime = '1575350530'

        This will set the next start time into the past.

      9. run

        php admin/tool/task/cli/schedule_task.php --execute=\\core\\task\\automated_backup_task

        from the command line.

      10. Go to "Site administration > Reports > Backups" and check that the status is set to "Queued"
      11. run

        php admin/cli/cron.php

        to execute the adhoc tasks.

      12. Go to "Site administration > Reports > Backups" and check that the status is set to "OK"
      13. Head to the restore section of one of the courses (Course administration > Restore) and check that there is a backup file located in the "Automated backups" section.
      Show
      Create a few courses Purge caches Go to Administration > Site administration > Courses > Backups > Automated backup setup Set backup_auto_active to enabled Choose all days of the week run php admin/tool/task/cli/schedule_task.php --execute=\\core\\task\\automated_backup_task from the command line. Go to "Site administration > Reports > Backups" and check that the status is set to "Automated backup pending" run the following sql command on you database (requires direct access to the database) UPDATE "mdl_backup_courses" SET nextstarttime = '1575350530' This will set the next start time into the past. run php admin/tool/task/cli/schedule_task.php --execute=\\core\\task\\automated_backup_task from the command line. Go to "Site administration > Reports > Backups" and check that the status is set to "Queued" run php admin/cli/cron.php to execute the adhoc tasks. Go to "Site administration > Reports > Backups" and check that the status is set to "OK" Head to the restore section of one of the courses (Course administration > Restore) and check that there is a backup file located in the "Automated backups" section.

    Description

      Currently the processing of automated backups runs a single long running cron task with its own bespoke locking.
      One problem here is in a Moodle site that has a lot of courses that change often, the rate of change of the courses can be to much for the current way of processing automated backups. This means that automated course backups "get behind" and backups don't happen in a timely manner.
      MDL-65201 is a first step to making this better by improving how courses are ordered for automated backups.
      This request is to further refactor the automated backup code to:

      • Split individual backups into ad-hoc tasks. This gives the ability to run several automated backups in parallel and increase the overall throughput of the processing.
      • Remove the custom locking that is in the automated backup code and make automated backups use the regular scheduled task locking.

      Attachments

        Issue Links

          Activity

            People

              yao9394 John Yao
              mattp@catalyst-au.net Matt Porritt
              Matt Porritt Matt Porritt
              Adrian Greeve Adrian Greeve
              Janelle Barcega Janelle Barcega
              Adrian Greeve, David Woloszyn, Huong Nguyen, Jake Dallimore, Meirza, Michael Hawkins, Raquel Ortega, Safat Shahin, Stevani Andolo
              Votes:
              2 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:
                15/Jun/20

                Time Tracking

                  Estimated:
                  Original Estimate - 0 minutes
                  0m
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 2 days, 3 hours, 26 minutes
                  2d 3h 26m