Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-58777

Upgrade to 3.3 does a bunch of really weird stuff to the events table (sometimes)

    XMLWordPrintable

    Details

    • Testing Instructions:
      Hide

      Cron Test

      1. Install Moodle from commit 06e3b6d8ba
      2. Checkout integration
      3. Run the upgrade
      4. Inspect the table task_adhoc
      5. Verify there is only 1 entry - this should have NULL in the customdata column and '\core\task\refresh_mod_calendar_events_task' in the classname column.
      6. Run cron
      7. Verify that 

        Refreshing events for assign
        		 
        Refreshing events for chat
        		 
        Refreshing events for choice
        		 
        Refreshing events for data
        		 
        Refreshing events for feedback
        		 
        Refreshing events for lesson
        		 
        Refreshing events for quiz
        		 
        Refreshing events for scorm
        		 
        Refreshing events for workshop

        Is present in the output

      1. Repeat these steps a bunch of times and make sure the output mentioned above is always present

      Upgrade path tests

      Setup

      Don't do this yet, these steps are referenced from later steps.

      1. A course with:
        • 2 groups, A and B
        • 4 users, x, y, z, w
        • x in group A
        • y in group B
        • z in both group A and B
        • w in no groups
      2. A assign, lesson, and quiz modules (make them in that order) such that:
        • The "due" is date set (just set them to today's date (i.e., May at the time of writing), disable any other dates as well):
          • For lesson this is the "Deadline" date
          • For assign this is the "Due date" date
          • For quiz this is the "Close the quiz" date
        • The have names like "Test Lesson 1" to help keep track of things (we need to make more modules later))
      3. For each of the modules add (in this order):
        • A group override for the due date for group A (just change the month to June)
        • A group override for the due date for group B (just change the month to July)
        • A user override for the due date for user x (just change the month to August)
      4. The refreshevents.php script attached to this issue needs to be installed in your Moodle root

      3c45d26f58 → 5ccddd27df → 06e3b6d8ba → Integration

      1. Checkout and install Moodle from the 3c45d26f58 commit
      2. Go through the setup steps
      3. Checkout 5ccddd27df and run the upgrade script (NB do it using the PHP script,not MDK, since MDK does tricks where it checks out master or something)
      4. Inspect the task_adhoc table
      5. Verify that the task refresh_mod_calendar_events_task is queued twice (we can't fix this without rewriting history)
      6. Delete the refresh_mod_calendar_events_task records from the task_adhoc table
      7. Create a second set of modules as per the setup instructions (overrides and all), but call them things like "Test Lesson 2" (be careful with assign, there's a new date field "gradingdue", turn that off to not get confused)
      8. Inspect the event table and verify the following is true:
        • The Test Assignment 1 group A override has 1 in the priority column
        • The Test Assignment 1 group B override has 2 in the priority column
        • The Test Assignment 1 user override has 9999999 in the priority column (tThe reason this happens for assign and not the others is because we left behind an upgrade step in assign (seemingly by mistake, as the refresh events task will overwrite these changes when it runs anyway))
        • All the other "Test XXX 1" events have NULL in the priority column
        • The order for the "Test XXX 2" events will be kind of screwy, but make sure that the user overrides have priority 9999999 and the group ones have 1 for group A and 2 for group B (this is actually backwards to how it should be and gets fixed later) and the "original" event has NULL
      9. Run the refreshevents script attached to this issue (this is necessary due to a bug in the refresh events task)
      10. Inspect the table again, the order will still be all messed up and weird but just verify that all the "Test XXX 1" events have priority NULL, user overrides have 9999999 and the group overrides have 1 and 2 as explained before, the "Test XXX 2" events will be completely messed up with priorities all over the place - oh and the assignment overrides get nuked too - I don't know why (they should come back later). It's weird I know.
      11. Checkout 06e3b6d8ba and run the upgrade
      12. Inspect the task_adhoc table
      13. Verify there are two records, one will have the customdata field set and the other won't
      14. Delete the tasks from the task_adhoc table
      15. Create a new set of events as before, but use titles like "Test Lesson 3"
      16. Inspect the event table and verify the following is true (the order will still be whacky):
        • The events created prior to the upgrade haven't changed
        • The priorities for all the new "original" events are NULL
        • The priorities for all the new assign group overrides are 1 for group A and 2 for group B
        • The prioritieis for all the new group overrides for lesson and quiz are 2 for group A and 1 for group B
        • All the new user overrides have priority 0
      17. Run the refreshevents script
      18. Inspect the event table (sorting by name might be useful since the IDs are going to be all over the place) and verify the following is true:
        • All the events for each module exist (original, 2 group override events and a user override - assign will also have a new "gradingdue" event)
        • The priorities for all the assign group overrides are 1 for group A and 2 for group B
        • The prioritieis for all the group overrides for lesson and quiz are 2 for group A and 1 for group B
        • The priorities for the user overrides (and possibly the original module events) are a mix of 0 and 9999999 (this is the bug we're trying to fix)
      19. Checkout integration and run the upgrade
      20. Inspect the task_adhoc table
      21. Verify there is only one record
      22. Create a new set of events as before, but use titles like "Test Lesson 4"
      23. Inspect the event table and verify the following is true (sorting by ID again will probably help):
        • The events created prior to the upgrade haven't changed
        • The order of the new events (by module) is: Original event, override for A, override for B, user override
        • The priorities for all the new "original" events are NULL
        • The priorities for all the new assign group overrides are 1 for group A and 2 for group B
        • The prioritieis for all the new group overrides for lesson and quiz are 2 for group A and 1 for group B
        • All the new user overrides have priority 0
      24. Run admin/cli/cron.php. Up until now, we've been running the 'refreshevents' script, but now that we're on integration, the cron will work.
      25. Inspect the event table (sorting by name might be useful since the IDs are going to be all over the place) and verify the following is true:
        • All there are 4 events per module (with the exception of the first assign module and second assign module, since those had "gradingdue" events added)
        • All the "original" events have priority NULL
        • The order of the new events (by module) is: Original event, override for A, override for B, user override
        • The priorities for the "original" events are NULL (all of the events, not just the new ones)
        • The priorities for all the assign group overrides are 1 for group A and 2 for group B (all of the events, not just the new ones)
        • The prioritieis for all thew group overrides for lesson and quiz are 2 for group A and 1 for group B (all of the events, not just the new ones)
        • All the new user overrides have priority 0 (all of the events, not just the new ones)
      26. Sanity check:
        • Log in as x and view the dashboard
        • Verify all the events show the user override (months will be set to August)
        • Log in as y
        • Verify all the events are the group B overrides (months will be set to July)
        • Log in as z
        • Verify all the events for assign modules are the group A overrides (months will be set to June)
        • Verify all the events for lesson and quiz show the group B overrides (months will be set to July)
        • Log in as _w-
        • Verify all the overrides are the original due dates (months will be set to May)
        • Verify you don't see the quiz events at all (quiz sets the close date to the current date plus time, so by the time you get to this step, enough time will have passed that the event is over)

      3c45d26f58 → 06e3b6d8ba → Integration

      1. Checkout and install Moodle from the 3c45d26f58 commit
      2. Go through the setup steps
      3. Checkout 06e3b6d8ba and run the upgrade script (NB do it using the PHP script,not MDK, since MDK does tricks where it checks out master or something)
      4. Inspect the task_adhoc table
      5. Verify that the task refresh_mod_calendar_events_task is queued thrice (we can't fix this without rewriting history)
      6. Delete the refresh_mod_calendar_events_task records from the task_adhoc table
      7. Create a second set of modules as per the setup instructions (overrides and all), but call them things like "Test Lesson 2" (be careful with assign, there's a new date field "gradingdue", turn that off to not get confused)
      8. Inspect the event table and verify the following is true:
        • The Test Assignment 1 group A override has 1 in the priority column
        • The Test Assignment 1 group B override has 2 in the priority column
        • The Test Assignment 1 user override has 9999999 in the priority column (the reason this happens for assign and not the others is because we left behind an upgrade step in assign (seemingly by mistake, as the refresh events task will overwrite these changes when it runs anyway))
        • All the other "Test XXX 1" events have NULL in the priority column
        • The order for the "Test XXX 2" events will be kind of screwy, but make sure that the user overrides have priority 0 and the group ones have 2 for group A and 1 for group B
      9. Inspect the table again, the order will still be all messed up and weird (and a "gradingdue" event will show up)but just verify that all the following is true:
        • The priorities for all the assign group overrides are 1 for group A and 2 for group B
        • The prioritieis for all the group overrides for lesson and quiz are 2 for group A and 1 for group B
        • The priorities for the user overrides are a mix of 0 and NULL (this is the bug we're trying to fix)
      10. Checkout integration and run the upgrade
      11. Inspect the task_adhoc table
      12. Verify there is only one refresh events record
      13. Create a new set of events as before, but use titles like "Test Lesson 3"
      14. Inspect the event table and verify the following is true (sorting by ID again will probably help):
        • The events created prior to the upgrade haven't changed
        • The order of the new events (by module) is: Original event, override for A, override for B, user override
        • The priorities for all the new "original" events are NULL
        • The priorities for all the new assign group overrides are 1 for group A and 2 for group B
        • The prioritieis for all the new group overrides for lesson and quiz are 2 for group A and 1 for group B
        • All the new user overrides have priority 0
      15. Run the refreshevents script
      16. Inspect the event table (sorting by name might be useful since the IDs are going to be all over the place) and verify the following is true:
        • All there are 4 events per module (with the exception of the first assign - it will get a "gradingdue" event added)
        • All the "original" events have priority NULL
        • The order of the new events (by module) is: Original event, override for A, override for B, user override
        • The priorities for the "original" events are NULL (all of the events, not just the new ones)
        • The priorities for all the assign group overrides are 1 for group A and 2 for group B (all of the events, not just the new ones)
        • The prioritieis for all thew group overrides for lesson and quiz are 2 for group A and 1 for group B (all of the events, not just the new ones)
        • All the new user overrides have priority 0 (all of the events, not just the new ones)
      17. Do the sanity check from before

      3c45d26f58 → Integration

      1. Checkout and install Moodle from the 3c45d26f58 commit
      2. Go through the setup steps
      3. Checkout integration and run the upgrade script
      4. Inspect the task_adhoc table
      5. Verify that the task refresh_mod_calendar_events_task is once
      6. Create a second set of modules as per the setup instructions (overrides and all), but call them things like "Test Lesson 2" (be careful with assign, there's a new date field "gradingdue", turn that off to not get confused)
      7. Inspect the event table and verify the following is true:
        • The events made before the upgrade all have NULL in the priority column
        • The priorities for all the new "original" events are NULL
        • The priorities for all the new assign group overrides are 1 for group A and 2 for group B
        • The prioritieis for all the new group overrides for lesson and quiz are 2 for group A and 1 for group B
        • All the new user overrides have priority 0
      8. Run the refreshevents script
      9. Inspect the event table and verify the following is true:
        • All there are 4 events per module (with the exception of the first assign module - it will get "gradingdue" event added)
        • All the "original" events have priority NULL
        • The order of the new events (by module) is: Original event, override for A, override for B, user override
        • The priorities for the "original" events are NULL (all of the events, not just the new ones)
        • The priorities for all the assign group overrides are 1 for group A and 2 for group B (all of the events, not just the new ones)
        • The prioritieis for all thew group overrides for lesson and quiz are 2 for group A and 1 for group B (all of the events, not just the new ones)
        • All the new user overrides have priority 0 (all of the events, not just the new ones)
      10. Do the sanity check from before

      If you'd really like to, you can test 5ccddd27df → Integration as it's the only path not tested by these steps. It should be similar to 3c45d26f58 → Integration except initially events will be created with priority 9999999 for user overrides, and the priority of events for lesson and quiz will be backswards (1 for A and 2 for B).

      Show
      Cron Test Install Moodle from commit 06e3b6d8ba Checkout integration Run the upgrade Inspect the table task_adhoc Verify there is only 1 entry - this should have NULL in the customdata column and '\core\task\refresh_mod_calendar_events_task' in the classname column. Run cron Verify that Refreshing events for assign Refreshing events for chat Refreshing events for choice Refreshing events for data Refreshing events for feedback Refreshing events for lesson Refreshing events for quiz Refreshing events for scorm Refreshing events for workshop Is present in the output Repeat these steps a bunch of times and make sure the output mentioned above is always present Upgrade path tests Setup Don't do this yet , these steps are referenced from later steps. A course with: 2 groups, A and B 4 users, x , y , z , w x in group A y in group B z in both group A and B w in no groups A assign, lesson, and quiz modules (make them in that order) such that: The "due" is date set (just set them to today's date (i.e., May at the time of writing), disable any other dates as well): For lesson this is the "Deadline" date For assign this is the "Due date" date For quiz this is the "Close the quiz" date The have names like "Test Lesson 1" to help keep track of things (we need to make more modules later)) For each of the modules add (in this order): A group override for the due date for group A (just change the month to June) A group override for the due date for group B (just change the month to July) A user override for the due date for user x (just change the month to August) The refreshevents.php script attached to this issue needs to be installed in your Moodle root 3c45d26f58 → 5ccddd27df → 06e3b6d8ba → Integration Checkout and install Moodle from the 3c45d26f58 commit Go through the setup steps Checkout 5ccddd27df and run the upgrade script ( NB do it using the PHP script, not MDK, since MDK does tricks where it checks out master or something) Inspect the task_adhoc table Verify that the task refresh_mod_calendar_events_task is queued twice (we can't fix this without rewriting history) Delete the refresh_mod_calendar_events_task records from the task_adhoc table Create a second set of modules as per the setup instructions (overrides and all), but call them things like "Test Lesson 2" (be careful with assign, there's a new date field "gradingdue", turn that off to not get confused) Inspect the event table and verify the following is true: The Test Assignment 1 group A override has 1 in the priority column The Test Assignment 1 group B override has 2 in the priority column The Test Assignment 1 user override has 9999999 in the priority column (tThe reason this happens for assign and not the others is because we left behind an upgrade step in assign (seemingly by mistake, as the refresh events task will overwrite these changes when it runs anyway)) All the other "Test XXX 1" events have NULL in the priority column The order for the "Test XXX 2" events will be kind of screwy, but make sure that the user overrides have priority 9999999 and the group ones have 1 for group A and 2 for group B (this is actually backwards to how it should be and gets fixed later) and the "original" event has NULL Run the refreshevents script attached to this issue (this is necessary due to a bug in the refresh events task) Inspect the table again, the order will still be all messed up and weird but just verify that all the "Test XXX 1" events have priority NULL, user overrides have 9999999 and the group overrides have 1 and 2 as explained before, the "Test XXX 2" events will be completely messed up with priorities all over the place - oh and the assignment overrides get nuked too - I don't know why (they should come back later). It's weird I know. Checkout 06e3b6d8ba and run the upgrade Inspect the task_adhoc table Verify there are two records, one will have the customdata field set and the other won't Delete the tasks from the task_adhoc table Create a new set of events as before, but use titles like "Test Lesson 3" Inspect the event table and verify the following is true (the order will still be whacky): The events created prior to the upgrade haven't changed The priorities for all the new "original" events are NULL The priorities for all the new assign group overrides are 1 for group A and 2 for group B The prioritieis for all the new group overrides for lesson and quiz are 2 for group A and 1 for group B All the new user overrides have priority 0 Run the refreshevents script Inspect the event table (sorting by name might be useful since the IDs are going to be all over the place) and verify the following is true: All the events for each module exist (original, 2 group override events and a user override - assign will also have a new "gradingdue" event) The priorities for all the assign group overrides are 1 for group A and 2 for group B The prioritieis for all the group overrides for lesson and quiz are 2 for group A and 1 for group B The priorities for the user overrides (and possibly the original module events) are a mix of 0 and 9999999 (this is the bug we're trying to fix) Checkout integration and run the upgrade Inspect the task_adhoc table Verify there is only one record Create a new set of events as before, but use titles like "Test Lesson 4" Inspect the event table and verify the following is true (sorting by ID again will probably help): The events created prior to the upgrade haven't changed The order of the new events (by module) is: Original event, override for A, override for B, user override The priorities for all the new "original" events are NULL The priorities for all the new assign group overrides are 1 for group A and 2 for group B The prioritieis for all the new group overrides for lesson and quiz are 2 for group A and 1 for group B All the new user overrides have priority 0 Run admin/cli/cron.php. Up until now, we've been running the 'refreshevents' script, but now that we're on integration, the cron will work. Inspect the event table (sorting by name might be useful since the IDs are going to be all over the place) and verify the following is true: All there are 4 events per module (with the exception of the first assign module and second assign module, since those had "gradingdue" events added) All the "original" events have priority NULL The order of the new events (by module) is: Original event, override for A, override for B, user override The priorities for the "original" events are NULL (all of the events, not just the new ones) The priorities for all the assign group overrides are 1 for group A and 2 for group B (all of the events, not just the new ones) The prioritieis for all thew group overrides for lesson and quiz are 2 for group A and 1 for group B (all of the events, not just the new ones) All the new user overrides have priority 0 (all of the events, not just the new ones) Sanity check: Log in as x and view the dashboard Verify all the events show the user override (months will be set to August) Log in as y Verify all the events are the group B overrides (months will be set to July) Log in as z Verify all the events for assign modules are the group A overrides (months will be set to June) Verify all the events for lesson and quiz show the group B overrides (months will be set to July) Log in as _w- Verify all the overrides are the original due dates (months will be set to May) Verify you don't see the quiz events at all (quiz sets the close date to the current date plus time, so by the time you get to this step, enough time will have passed that the event is over) 3c45d26f58 → 06e3b6d8ba → Integration Checkout and install Moodle from the 3c45d26f58 commit Go through the setup steps Checkout 06e3b6d8ba and run the upgrade script ( NB do it using the PHP script, not MDK, since MDK does tricks where it checks out master or something) Inspect the task_adhoc table Verify that the task refresh_mod_calendar_events_task is queued thrice (we can't fix this without rewriting history) Delete the refresh_mod_calendar_events_task records from the task_adhoc table Create a second set of modules as per the setup instructions (overrides and all), but call them things like "Test Lesson 2" (be careful with assign, there's a new date field "gradingdue", turn that off to not get confused) Inspect the event table and verify the following is true: The Test Assignment 1 group A override has 1 in the priority column The Test Assignment 1 group B override has 2 in the priority column The Test Assignment 1 user override has 9999999 in the priority column (the reason this happens for assign and not the others is because we left behind an upgrade step in assign (seemingly by mistake, as the refresh events task will overwrite these changes when it runs anyway)) All the other "Test XXX 1" events have NULL in the priority column The order for the "Test XXX 2" events will be kind of screwy, but make sure that the user overrides have priority 0 and the group ones have 2 for group A and 1 for group B Inspect the table again, the order will still be all messed up and weird (and a "gradingdue" event will show up)but just verify that all the following is true: The priorities for all the assign group overrides are 1 for group A and 2 for group B The prioritieis for all the group overrides for lesson and quiz are 2 for group A and 1 for group B The priorities for the user overrides are a mix of 0 and NULL (this is the bug we're trying to fix) Checkout integration and run the upgrade Inspect the task_adhoc table Verify there is only one refresh events record Create a new set of events as before, but use titles like "Test Lesson 3" Inspect the event table and verify the following is true (sorting by ID again will probably help): The events created prior to the upgrade haven't changed The order of the new events (by module) is: Original event, override for A, override for B, user override The priorities for all the new "original" events are NULL The priorities for all the new assign group overrides are 1 for group A and 2 for group B The prioritieis for all the new group overrides for lesson and quiz are 2 for group A and 1 for group B All the new user overrides have priority 0 Run the refreshevents script Inspect the event table (sorting by name might be useful since the IDs are going to be all over the place) and verify the following is true: All there are 4 events per module (with the exception of the first assign - it will get a "gradingdue" event added) All the "original" events have priority NULL The order of the new events (by module) is: Original event, override for A, override for B, user override The priorities for the "original" events are NULL (all of the events, not just the new ones) The priorities for all the assign group overrides are 1 for group A and 2 for group B (all of the events, not just the new ones) The prioritieis for all thew group overrides for lesson and quiz are 2 for group A and 1 for group B (all of the events, not just the new ones) All the new user overrides have priority 0 (all of the events, not just the new ones) Do the sanity check from before 3c45d26f58 → Integration Checkout and install Moodle from the 3c45d26f58 commit Go through the setup steps Checkout integration and run the upgrade script Inspect the task_adhoc table Verify that the task refresh_mod_calendar_events_task is once Create a second set of modules as per the setup instructions (overrides and all), but call them things like "Test Lesson 2" (be careful with assign, there's a new date field "gradingdue", turn that off to not get confused) Inspect the event table and verify the following is true: The events made before the upgrade all have NULL in the priority column The priorities for all the new "original" events are NULL The priorities for all the new assign group overrides are 1 for group A and 2 for group B The prioritieis for all the new group overrides for lesson and quiz are 2 for group A and 1 for group B All the new user overrides have priority 0 Run the refreshevents script Inspect the event table and verify the following is true: All there are 4 events per module (with the exception of the first assign module - it will get "gradingdue" event added) All the "original" events have priority NULL The order of the new events (by module) is: Original event, override for A, override for B, user override The priorities for the "original" events are NULL (all of the events, not just the new ones) The priorities for all the assign group overrides are 1 for group A and 2 for group B (all of the events, not just the new ones) The prioritieis for all thew group overrides for lesson and quiz are 2 for group A and 1 for group B (all of the events, not just the new ones) All the new user overrides have priority 0 (all of the events, not just the new ones) Do the sanity check from before If you'd really like to, you can test 5ccddd27df → Integration as it's the only path not tested by these steps. It should be similar to 3c45d26f58 → Integration except initially events will be created with priority 9999999 for user overrides, and the priority of events for lesson and quiz will be backswards (1 for A and 2 for B).
    • Affected Branches:
      MOODLE_33_STABLE
    • Fixed Branches:
      MOODLE_33_STABLE
    • Pull from Repository:
    • Pull Master Branch:
      MDL-58777-master
    • Pull Master Diff URL:

      Description

      While Ryan was working on MDL-58518 it was discovered that the upgrade code for the events table is very flaky. We identified four problems:

      1. The refresh events task gets queued twice. We did attempt to stop this happening in MDL-58578 by adding a flag to the earliest instance of the task being queued but it looks like there was an earlier one.
      2. The refresh events ad-hoc task is incredibly flaky. It uses function_exists to determine whether or not to call the refresh events function from modules, but it does not include the module's lib.php - so depending on the order cron does stuff, the events may never get refreshed and the task will be dequeued forever.
      3. The code for refresh events in lesson and quiz does not explicitly set the priority for the "default" module event, and our DML sets it to 0 - which has a different meaning to null for us in this case. We must explicitly set it to null.
      4. There still appears to be some code in assign using 9999999 for priority stuff when it should be using 0 (nevermind it's something else)

        Attachments

          Issue Links

          Testing discovered

          Bug - A problem which impairs or prevents Moodle from functioning correctly. MDL-58900 Possible to create two assign overrides with the same sortorder

          • Minor - Minor loss of function where workaround is possible
          • Closed

            Activity

              People

              Assignee:
              cameron1729 cameron1729
              Reporter:
              cameron1729 cameron1729
              Peer reviewer:
              Ryan Wyllie
              Integrator:
              Jake Dallimore
              Tester:
              Mark Nelson
              Participants:
              Component watchers:
              Adrian Greeve, Jake Dallimore, Mathew May, Mihail Geshoski, Peter Dias, Andrew Nicols, Jun Pataleta, Michael Hawkins, Shamim Rezaie, Simey Lameze, Adrian Greeve, Jake Dallimore, Mathew May, Mihail Geshoski, Peter Dias, Tim Hunt, Andrew Nicols, Jun Pataleta, Michael Hawkins, Shamim Rezaie, Simey Lameze
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:
                Fix Release Date:
                15/May/17