Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-75435

Improve the performance of the automated backups manager task by not reading from the standard log

XMLWordPrintable

    • MDL-75435-main
    • Hide

      Testing instructions:

       

      Set up: 

      1. Set backup | backup_auto_active to Enabled.
      2. Tick all days for the backup | backup_auto_weekdays setting.
      3. Set backup | backup_auto_skip_modif_days to 10 days.
      4. Set backup | backup_auto_skip_modif_prev to Yes.

      Test 1

      1. Create a course.
      2. Check the course id.
      3. Execute automated_backup_task  

        php admin/cli/scheduled_task.php --execute="core\task\automated_backup_task"

      4. Confirm you get the following message  "Skipping course id [CourseID]: Not scheduled for backup until [Date] "
      5. Change your date system to match it with the previous given date. 
      6. Execute again "automated_backup_task" (Step 3)
      7. Confirm, that the course is scheduled for backup, the following message should be displayed: ""Putting backup of course id [CourseID] in adhoc task queue"
      8. Run adhoc tasks to actually back up the course by executing  "php admin/cli/adhoc_task.php -e"
      9. Execute again "automated_backup_task" (Step 3)
      10. Confirm, that the course is skipped as not modified, the following message should be displayed: "Skipping course id [CourseID]: Not modified since previous backup "
      11. Create a user.
      12. Enrol the user to the course.
      13. Execute "automated_backup_task" (Step 3)
      14. Confirm, that the course is scheduled for backup, and the following message is displayed: "Putting backup of course id [CourseID] in adhoc task queue"
      15. Run adhoc tasks to actually back up the course by executing  (Step 8).
      Show
      Testing instructions:   Set up:  Set backup | backup_auto_active to Enabled . Tick all days for the backup | backup_auto_weekdays setting. Set backup | backup_auto_skip_modif_days to 10 days. Set backup | backup_auto_skip_modif_prev to Yes . Test 1 Create a course. Check the course id. Execute automated_backup_task  php admin/cli/scheduled_task.php --execute= "core\task\automated_backup_task" Confirm you get the following message  "Skipping course id [CourseID] : Not scheduled for backup until [Date] " Change your date system to match it with the previous given date.  Execute again "automated_backup_task" (Step 3) Confirm , that the course is scheduled for backup, the following message should be displayed: ""Putting backup of course id [CourseID] in adhoc task queue" Run adhoc tasks to actually back up the course by executing  "php admin/cli/adhoc_task.php -e" Execute again "automated_backup_task" (Step 3) Confirm , that the course is skipped as not modified, the following message should be displayed: "Skipping course id [CourseID] : Not modified since previous backup " Create a user. Enrol the user to the course. Execute "automated_backup_task" (Step 3) Confirm , that the course is scheduled for backup, and the following message is displayed: "Putting backup of course id [CourseID] in adhoc task queue" Run adhoc tasks to actually back up the course by executing  (Step 8).
    • Hide

      Code verified against automated checks with warnings.

      Checked MDL-75435 using repository: https://github.com/raortegar/moodle.git

      Should these errors be fixed?

      Built on: Wed 11 Dec 2024 11:46:21 AM UTC

      Show
      Code verified against automated checks with warnings. Checked MDL-75435 using repository: https://github.com/raortegar/moodle.git main (0 errors / 3 warnings) [branch: MDL-75435-main | CI Job ] overview (0/0) , phplint (0/0) , phpcs (0/0) , js (0/0) , css (0/0) , phpdoc (0/0) , commit (0/0) , savepoint (0/0) , thirdparty (0/0) , externalbackup (0/3) , grunt (0/0) , shifter (0/0) , mustache (0/0) , gherkin (0/0) , Should these errors be fixed? Built on: Wed 11 Dec 2024 11:46:21 AM UTC

      This is the root cause here where it queries the log, and does that query inside a loop once per course. With a huge number of courses and a huge log table this is huge bottle neck.

      https://github.com/moodle/moodle/blob/master/backup/util/helper/backup_cron_helper.class.php#L780-L793

      I'm proposing to radically simply this to:

      1. have a moodle event handler looking for any changes to course events
      2. if found, then just set a 'course has modifications' flag
      3. the backup tasks will look at this flag when scheduling the automated backups and reset the flag when they are done

      This change depends on MDL-73926, so the commit is based on it. It will be rebased once MDL-73926 gets accepted.

            raquel.ortega@moodle.com Raquel Ortega
            brendanheywood Brendan Heywood
            Brendan Heywood Brendan Heywood
            Votes:
            1 Vote for this issue
            Watchers:
            13 Start watching this issue

              Created:
              Updated:

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0 minutes
                0m
                Logged:
                Time Spent - 2 days, 6 hours, 14 minutes
                2d 6h 14m

                  Error rendering 'clockify-timesheets-time-tracking-reports:timer-sidebar'. Please contact your Jira administrators.