Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-71156

Upgrade step from MDL-67494 corrupts calendar events

    XMLWordPrintable

    Details

    • Testing Instructions:
      Hide

      Setup (before the patch!)

      1. Clone a site (but don't install yet) and checkout to the version released before MDL-67494 (e.g. for master: "git checkout 95dd305cc6"):
        Version Commit hash
        master 95dd305cc6
        3.11 4652b66ee4
        3.10 3799c7f535
        3.9 f32754319e
      2. Proceed with the install now.
      3. Log in as admin
      4. Go to Site administration > Courses > Backups > General import defaults, enable Allow admin conflict resolution. Save the changes.
      5. Restore the backup-moodle2-course-5-testing_mdl-71156-20210409-1910.mbz that adds a course, with some enrolled users and groups. Note that action events, user and group overrides will be restored in the backup file.
      6. Create other events:
        • Standard - user, site, category, course, group events
        • Subscription - create a calendar subscription using the following iCal link:

          https://calendar.google.com/calendar/ical/en.usa%23holiday%40group.v.calendar.google.com/public/basic.ics
          

          Set the event type of these events from the subscription as Site events.

      7. Now checkout to the version with MDL-67494 code:
        Version Commit hash
        master 96b49ddc
        3.11 f23ffe66
        3.10 c6f3ecb4
        3.9 329a5109
      8. Upgrade your moodle instance again.

      Upgrade step and ad-hoc test (after the patch!)

      1. Check out the latest version for your moodle instance (e.g. for master "git checkout master")
      2. Run upgrade.
      3. Make sure a new record has mdl_task_adhoc for \core\task\calendar_fix_orphaned_events adhoc task.
      4. Run the command php admin/cli/adhoc_task.php --execute
      5. Make sure the adhoc is executed and displays a summary with the number of affected events, similarly to this one (with different numbers, of course):

        Execute adhoc task: core\task\calendar_fix_orphaned_events
        ... started 17:05:53. Current memory use 16.6MB.
        Totals: 17 / 16 (total / wrong)
          - standards events: 5 / 4 (group: 1, course: 1, category: 1, site: 1)
          - subscription events: 0 / 0 - all good!
          - override events: 9 / 9 (quiz: 5, lesson: 4)
          - action events: 3 / 3 (quiz: 1, lesson: 2)
          - other events: 0 / 0 - all good!
         
        Processing override events
        ..
        Finished!
         
        Processing standard events
        ....
        Finished!
         
        Processing action events
        Finished!
         
        ... used 94 dbqueries
        ... used 0.15929698944092 seconds
        Adhoc task complete: core\task\calendar_fix_orphaned_events
        

      6. Run the command

         php admin/cli/fix_orphaned_calendar_events.php 

      7. Make sure it displays 0 as the number of wrong events, eg Totals: 17 / 0 (total / wrong)

      Setup again (before the patch!)

      1. Repeat all the step in the first "Setup" section, creating a site with events and restoring the backup file. Before the patch!

      Testing ad-hoc task runtime (after the patch)

      1. Check out the latest version for your moodle instance.
      2. Open you config.php and add "$CFG->calendareventsmaxseconds = -1;" and save.
      3. Run the upgrade.
      4. Run the command: php admin/cli/adhoc_task.php --execute
      5. Make sure that after it started processing of events, it stops the execution showing the pending message and on which type of event, eg:

        Remaining EVENTTYPE events pending
        

      6. Run the same command again and confirm it display the pending message until it finishes fixing all events.

      CLI script

      1. Restore the backup with the calendar events.
      2. On your database, run the UPDATE query to simulate the existence of problematic events.

        UPDATE mdl_event SET userid = 0 WHERE eventtype <> 'user' OR priority <> 0

      3. On your terminal, run the following command:

         php admin/cli/fix_orphaned_calendar_events.php 

      4. Make sure a summary with the number of affected events is displayed, similarly to this one (with different numbers, of course). e.g. in master:

        == Checking the site status ==
         
        This site has executed the problematic upgrade step 2021052500.65 present in 4.0dev.
        This site has executed the fix upgrade step 2021052500.85 present in 4.0dev.
         
        == Checking the calendar events status ==
         
        Totals: 17 / 16 (total / wrong)
          - standards events: 5 / 4 (group: 1, course: 1, category: 1, site: 1)
          - subscription events: 0 / 0 - all good!
          - override events: 9 / 9 (lesson: 4, quiz: 5)
          - action events: 3 / 3 (lesson: 2, quiz: 1)
          - other events: 0 / 0 - all good!
         
        This site NEEDS to run the calendar events fix!
        To fix the calendar events, re-run this script with the --fix option.
        

      5. Run the command again, with the --fix option this time.
      6. It will display a confirmation message, type Y to start the execution.
      7. Make sure the script fixes those orphaned events and display a summary like this one:

        == Fixing as many as possible calendar events ==
         
        Processing override events
        ..
        Finished!
         
        Processing standard events
        ....
        Finished!
         
        Processing action events
        Finished!
         
        == Checking the calendar events status (after fix) ==
         
        Totals: 17 / 0 (total / wrong)
          - standards events: 5 / 0 - all good!
          - subscription events: 0 / 0 - all good!
          - override events: 9 / 0 - all good!
          - action events: 3 / 0 - all good!
          - other events: 0 / 0 - all good!
        

        If you compare the summary before/after you can see it has 0 calendar events showing as wrong.

      Show
      Setup (before the patch!) Clone a site (but don't install yet) and checkout to the version released before MDL-67494 (e.g. for master: " git checkout 95dd305cc6 "): Version Commit hash master 95dd305cc6 3.11 4652b66ee4 3.10 3799c7f535 3.9 f32754319e Proceed with the install now. Log in as admin Go to Site administration > Courses > Backups > General import defaults , enable Allow admin conflict resolution . Save the changes. Restore the backup-moodle2-course-5-testing_mdl-71156-20210409-1910.mbz that adds a course, with some enrolled users and groups. Note that action events, user and group overrides will be restored in the backup file. Create other events: Standard - user, site, category, course, group events Subscription - create a calendar subscription using the following iCal link: https://calendar.google.com/calendar/ical/en.usa%23holiday%40group.v.calendar.google.com/public/basic.ics Set the event type of these events from the subscription as Site events. Now checkout to the version with MDL-67494 code: Version Commit hash master 96b49ddc 3.11 f23ffe66 3.10 c6f3ecb4 3.9 329a5109 Upgrade your moodle instance again. Upgrade step and ad-hoc test (after the patch!) Check out the latest version for your moodle instance (e.g. for master " git checkout master ") Run upgrade. Make sure a new record has mdl_task_adhoc for \core\task\calendar_fix_orphaned_events adhoc task. Run the command php admin/cli/adhoc_task.php --execute Make sure the adhoc is executed and displays a summary with the number of affected events, similarly to this one (with different numbers, of course): Execute adhoc task: core\task\calendar_fix_orphaned_events ... started 17:05:53. Current memory use 16.6MB. Totals: 17 / 16 (total / wrong) - standards events: 5 / 4 (group: 1, course: 1, category: 1, site: 1) - subscription events: 0 / 0 - all good! - override events: 9 / 9 (quiz: 5, lesson: 4) - action events: 3 / 3 (quiz: 1, lesson: 2) - other events: 0 / 0 - all good!   Processing override events .. Finished!   Processing standard events .... Finished!   Processing action events Finished!   ... used 94 dbqueries ... used 0.15929698944092 seconds Adhoc task complete: core\task\calendar_fix_orphaned_events Run the command php admin/cli/fix_orphaned_calendar_events.php Make sure it displays 0 as the number of wrong events, eg Totals: 17 / 0 (total / wrong) Setup again (before the patch!) Repeat all the step in the first "Setup" section, creating a site with events and restoring the backup file. Before the patch! Testing ad-hoc task runtime (after the patch) Check out the latest version for your moodle instance. Open you config.php and add " $CFG->calendareventsmaxseconds = -1;"  and save. Run the upgrade. Run the command: php admin/cli/adhoc_task.php --execute Make sure that after it started processing of events, it stops the execution showing the pending message and on which type of event, eg: Remaining EVENTTYPE events pending Run the same command again and confirm it display the pending message until it finishes fixing all events. CLI script Restore the backup with the calendar events. On your database, run the UPDATE query to simulate the existence of problematic events. UPDATE mdl_event SET userid = 0 WHERE eventtype <> 'user' OR priority <> 0 On your terminal, run the following command: php admin/cli/fix_orphaned_calendar_events.php Make sure a summary with the number of affected events is displayed, similarly to this one (with different numbers, of course). e.g. in master: == Checking the site status ==   This site has executed the problematic upgrade step 2021052500.65 present in 4.0dev. This site has executed the fix upgrade step 2021052500.85 present in 4.0dev.   == Checking the calendar events status ==   Totals: 17 / 16 (total / wrong) - standards events: 5 / 4 (group: 1, course: 1, category: 1, site: 1) - subscription events: 0 / 0 - all good! - override events: 9 / 9 (lesson: 4, quiz: 5) - action events: 3 / 3 (lesson: 2, quiz: 1) - other events: 0 / 0 - all good!   This site NEEDS to run the calendar events fix! To fix the calendar events, re-run this script with the --fix option. Run the command again, with the --fix option this time. It will display a confirmation message, type Y to start the execution. Make sure the script fixes those orphaned events and display a summary like this one: == Fixing as many as possible calendar events ==   Processing override events .. Finished!   Processing standard events .... Finished!   Processing action events Finished!   == Checking the calendar events status (after fix) ==   Totals: 17 / 0 (total / wrong) - standards events: 5 / 0 - all good! - subscription events: 0 / 0 - all good! - override events: 9 / 0 - all good! - action events: 3 / 0 - all good! - other events: 0 / 0 - all good! If you compare the summary before/after you can see it has 0 calendar events showing as wrong.
    • Affected Branches:
      MOODLE_310_STABLE, MOODLE_39_STABLE
    • Fixed Branches:
      MOODLE_310_STABLE, MOODLE_39_STABLE
    • Pull from Repository:
    • Pull 3.9 Branch:
    • Pull 3.10 Branch:
      MDL-71156-310
    • Pull 3.11 Branch:
      MDL-71156-311
    • Pull Master Branch:
      MDL-71156-master
    • Story Points:
      2
    • Sprint:
      Internationals - 3.11 Sprint 9

      Description

      The short version is that the upgrade step sets the userid to 0 for all events with an eventtype other than user (see here). But the problem is that overrides need a userid, and have an eventtype determined by the plugin (like open, close, or due). And the core code behavior is still correct after MDL-67494 (see here), but it's update step specifically corrupts those events.

      This specifically effect quiz, lesson, and assign. 

      Steps to reproduce:

      1. In 3.10.1, create a hidden quiz with a close date in the future.
      2. Add an override for that user changing the close date.
      3. Upgrade to 3.10.2
      4. Attempt to unhide the quiz
      5. You will get an invalid user error

       

      Basically the update step leaves override events with a userid, courseid, categoryid, and groupid all set to 0, which break various things (because of poor error handling in calendar, documented in MDL-70097 I think).

        Attachments

        1. backup-moodle2-course-5-testing_mdl-71156-20210409-1910.mbz
          223 kB
        2. image-2021-04-01-22-30-33-179.png
          120 kB
        3. image-2021-04-01-22-31-53-641.png
          33 kB
        4. MDL-71156_Test Passed.png
          326 kB
        5. moodle1.png
          14 kB
        6. moodle1-1.png
          14 kB
        7. moodle2.png
          moodle2.png
          28 kB
        8. moodle2-1.png
          28 kB
        9. Screenshot 2021-03-22 at 16.46.59.png
          37 kB

          Issue Links

            Activity

              People

              Assignee:
              lameze Simey Lameze
              Reporter:
              emerrill Eric Merrill
              Peer reviewer:
              Ferran Recio Ferran Recio
              Integrator:
              Jun Pataleta Jun Pataleta
              Tester:
              Gladys Basiana Gladys Basiana
              Participants:
              Component watchers:
              Andrew Lyons, Dongsheng Cai, Huong Nguyen, Jun Pataleta, Michael Hawkins, Shamim Rezaie, Simey Lameze
              Votes:
              51 Vote for this issue
              Watchers:
              87 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:
                Fix Release Date:
                10/May/21

                  Time Tracking

                  Estimated:
                  Original Estimate - 0 minutes
                  0m
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 2 weeks, 3 days, 9 minutes
                  2w 3d 9m