Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-65729

Ad-hoc task processing for automated backups

XMLWordPrintable

    • MOODLE_37_STABLE, MOODLE_38_STABLE
    • MOODLE_39_STABLE
    • master-MDL-65729
    • Hide
      1. Create a few courses
      2. Purge caches
      3. Go to Administration > Site administration > Courses > Backups > Automated backup setup
      4. Set backup_auto_active to enabled
      5. Choose all days of the week
      6. run 

        php admin/tool/task/cli/schedule_task.php --execute=\\core\\task\\automated_backup_task

        from the command line.

      7. Go to "Site administration > Reports > Backups" and check that the status is set to "Automated backup pending"
      8. run the following sql command on you database (requires direct access to the database) 

        UPDATE "mdl_backup_courses" SET nextstarttime = '1575350530'

        This will set the next start time into the past.

      9. run 

        php admin/tool/task/cli/schedule_task.php --execute=\\core\\task\\automated_backup_task

        from the command line.

      10. Go to "Site administration > Reports > Backups" and check that the status is set to "Queued"
      11. run 

        php admin/cli/cron.php

        to execute the adhoc tasks.

      12. Go to "Site administration > Reports > Backups" and check that the status is set to "OK"
      13. Head to the restore section of one of the courses (Course administration > Restore) and check that there is a backup file located in the "Automated backups" section.
      Show
      Create a few courses Purge caches Go to Administration > Site administration > Courses > Backups > Automated backup setup Set backup_auto_active to enabled Choose all days of the week run php admin/tool/task/cli/schedule_task.php --execute=\\core\\task\\automated_backup_task from the command line. Go to "Site administration > Reports > Backups" and check that the status is set to "Automated backup pending" run the following sql command on you database (requires direct access to the database) UPDATE "mdl_backup_courses" SET nextstarttime = '1575350530' This will set the next start time into the past. run php admin/tool/task/cli/schedule_task.php --execute=\\core\\task\\automated_backup_task from the command line. Go to "Site administration > Reports > Backups" and check that the status is set to "Queued" run php admin/cli/cron.php to execute the adhoc tasks. Go to "Site administration > Reports > Backups" and check that the status is set to "OK" Head to the restore section of one of the courses (Course administration > Restore) and check that there is a backup file located in the "Automated backups" section.

      Currently the processing of automated backups runs a single long running cron task with its own bespoke locking.
      One problem here is in a Moodle site that has a lot of courses that change often, the rate of change of the courses can be to much for the current way of processing automated backups. This means that automated course backups "get behind" and backups don't happen in a timely manner.
      MDL-65201 is a first step to making this better by improving how courses are ordered for automated backups.
      This request is to further refactor the automated backup code to:

      • Split individual backups into ad-hoc tasks. This gives the ability to run several automated backups in parallel and increase the overall throughput of the processing.
      • Remove the custom locking that is in the automated backup code and make automated backups use the regular scheduled task locking.

        1. screenshot-1.png
          screenshot-1.png
          285 kB

            yao9394 John Yao
            mattp@catalyst-au.net Matt Porritt
            Matt Porritt Matt Porritt
            Adrian Greeve Adrian Greeve
            Janelle Barcega Janelle Barcega
            Votes:
            2 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated:
              Resolved:

                Estimated:
                Original Estimate - 0 minutes
                0m
                Remaining:
                Remaining Estimate - 0 minutes
                0m
                Logged:
                Time Spent - 2 days, 3 hours, 26 minutes
                2d 3h 26m

                  Error rendering 'clockify-timesheets-time-tracking-reports:timer-sidebar'. Please contact your Jira administrators.