Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-61255

Write ad-hoc task to upgrade old messages to the new system

    XMLWordPrintable

    Details

    • Testing Instructions:
      Hide
      Test 1
      1. Install a fresh site on a checkout of a15c7459360f220a34c138f808340e3848c85b7c (before any new messaging changes landed).
      2. Run mdk users and mindev.
      3. Use the following script to generate messages for the users s1 to s5.

        <?php
         
        define('CLI_SCRIPT', true);
         
        require_once(__DIR__ . '/config.php');
        require_once($CFG->libdir . '/messagelib.php');
         
        $eventdata = new \core\message\message();
        $eventdata->fullmessageformat = FORMAT_PLAIN;
        $eventdata->notification = 0;
        $eventdata->component = 'moodle';
        $eventdata->name = 'instantmessage';
        $eventdata->courseid = $SITE->id;
         
        for ($i = 3; $i <= 7; $i++) {
            // Get the user.
            $userfrom = $DB->get_record('user', ['id' => $i]);
         
            $eventdata->userfrom = $userfrom;
         
            // Now, go through and send messages to other users.
            for ($k = 3; $k <= 7; $k++) {
                // Don't send to the same user.
                if ($i == $k) {
                    continue;
                }
         
                $userto = $DB->get_record('user', array('id' => $k));
         
                // Send 4 messages.
                for ($z = 1; $z <= 4; $z++) {
                    $eventdata->userto = $userto;
                    $eventdata->subject = 'This is test subject ' . $z;
                    $eventdata->smallmessage = 'This is test ' . $z . ' from ' . $userfrom->id . ' to ' . $userto->id;
                    $eventdata->fullmessage = 'This is test fullmessage ' . $z;
                    $eventdata->fullmessagehtml = '<p>' . $eventdata->fullmessage . '</p>';
                    $messageid = message_send($eventdata);
         
                    // Mark half the messages as read.
                    if ($z % 2 === 0) {
                        $message = $DB->get_record('message', ['id' => $messageid]);
                        message_mark_message_read($message, time());
                    }
                }
            }
        }
        

      4. Log in as each user from s1 to s5.
      5. Confirm you have 4 conversations (number next to the messaging icon).
      6. Confirm you have 2 unread messages from each user by clicking on the messaging icon (do not click on any of the individual conversations).
      7. Check that half the messages are in the 'message_read' table (make note of the ones that are).
      8. Once you have repeated the above steps for each user, log in as s5 and delete a message from each conversation (keep track of which messages were deleted).
      9. Now, log in as s7 and send a message to s8.
      10. Log in as s8, read the message from s7 and send a reply.
      11. Log in as s7 and send a reply to s8.
      12. Upgrade to integration master.
      13. Log in as each user from s1 to s5.
      14. Check you see that there are no messages.
      15. Go to the messaging index page and confirm there is a warning about the messages being temporarily unavailable.
      16. Run the cron.
      17. Log in as each user and confirm all is correct.
        • Confirm you have 4 conversations (number next to the messaging icon).
        • Confirm you have 2 unread messages from each user by clicking on the messaging icon (do not click on any of the individual conversations).
      18. Confirm the 'messages' table is populated with all the messages (there should be 80 messages).
      19. Confirm in 'message_user_actions' (action = '1') that half the messages are marked as read + all the messages received by s5 (since they looked at each conversation to delete a message - so should be another half + another 8 read messages in the table).
      20. Confirm in 'message_user_actions' (action = '2') that the messages you deleted were deleted.
      21. Log in as user s5 and confirm you don't see the messages you deleted before the migration (there will be 1 message missing from each conversation)
      22. Confirm the 'message' and 'message_read' tables only contain the 3 messages from s7 and s8
      Test 2
      1. Install a fresh site on a checkout of a15c7459360f220a34c138f808340e3848c85b7c (before any new messaging changes landed).
      2. Run mdk users and mindev.
      3. Use the following script to generate notifications for the users s1 to s5.

        <?php
         
        define('CLI_SCRIPT', true);
         
        require_once(__DIR__ . '/config.php');
        require_once($CFG->libdir . '/messagelib.php');
         
        $eventdata = new \core\message\message();
        $eventdata->modulename = 'assign';
        $eventdata->fullmessage = 'Full message for notification';
        $eventdata->fullmessageformat = FORMAT_PLAIN;
        $eventdata->fullmessagehtml = '<p>Full message HTML for notification</p>';
        $eventdata->smallmessage = 'This is test notification';
        $eventdata->name = 'assign_notification';
        $eventdata->component = 'mod_assign';
        $eventdata->notification = 1;
        $eventdata->contexturl = 'https://www.google.com';
        $eventdata->contexturlname = 'google';
        $eventdata->courseid = $SITE->id;
         
        for ($i = 3; $i <= 7; $i++) {
            // Get the user.
            $userfrom = $DB->get_record('user', ['id' => $i]);
         
            // Now, go through and send notifications to other users.
            for ($k = 3; $k <= 7; $k++) {
                // Don't send to the same user.
                if ($i == $k) {
                    continue;
                }
         
                $userto = $DB->get_record('user', array('id' => $k));
         
                // Set their preference to receive assign_notifications.
                set_user_preference('message_provider_mod_assign_assign_notification_loggedin', 'popup', $userto);
                set_user_preference('message_provider_mod_assign_assign_notification_loggedoff', 'popup', $userto);
         
                $eventdata->userfrom = $userfrom;
                $eventdata->userto = $userto;
                for ($z = 1; $z <= 4; $z++) {
                    $eventdata->subject = 'This is test notification ' . $z . ' from ' . $userfrom->id . ' to ' . $userto->id;
                    $messageid = message_send($eventdata);
         
         
                    // Mark half the messages as read.
                    if ($z % 2 === 0) {
                        $message = $DB->get_record('message', ['id' => $messageid]);
                        message_mark_message_read($message, time());
                    }
                }
            }
        }
        

      4. Log in as each user from s1 to s5.
      5. Confirm you have 16 notifications, 8 of which are read.
      6. Check that half the messages are in the 'message_read' table.
      7. Upgrade to integration master.
      8. Log in as each user from s1 to s5.
      9. Check you see that there are no notifications.
      10. Go to the notification index page (click 'See all' in notification popover) and confirm there is a warning about the notification being temporarily unavailable.
      11. Run the cron.
      12. Log in as each user and confirm all is correct.
      13. Confirm the 'notifications' table is populated with all the notifications (there should be 80 notifications) and half have the 'timeread' column set.
      Test 3

      This is to be done on integration master.

      1. Log in as an Admin.
      2. Create a course.
      3. Enrol a student.
      4. Create an assignment.
      5. Log in as the student.
      6. Click on ‘Notification preferences’ in the notification popover.
      7. Turn on all assignment notifications for the web processor.
      8. Submit an assignment.
      9. Confirm a notification is created in the notification popover.
      Test 4

      This is to be done on integration master.

      1. Log in as an Admin.
      2. Create a course.
      3. Enrol a student.
      4. Create an assignment.
      5. Log in as the student.
      6. Click on ‘Notification preferences’ in the notification popover.
      7. Turn off all assignment notifications for the web processor.
      8. Submit an assignment.
      9. Confirm no notification is created in the notification popover.
      Show
      Test 1 Install a fresh site on a checkout of a15c7459360f220a34c138f808340e3848c85b7c (before any new messaging changes landed). Run mdk users and mindev. Use the following script to generate messages for the users s1 to s5. <?php   define( 'CLI_SCRIPT' , true);   require_once (__DIR__ . '/config.php' ); require_once ( $CFG ->libdir . '/messagelib.php' );   $eventdata = new \core\message\message(); $eventdata ->fullmessageformat = FORMAT_PLAIN; $eventdata ->notification = 0; $eventdata ->component = 'moodle' ; $eventdata ->name = 'instantmessage' ; $eventdata ->courseid = $SITE ->id;   for ( $i = 3; $i <= 7; $i ++) { // Get the user. $userfrom = $DB ->get_record( 'user' , [ 'id' => $i ]);   $eventdata ->userfrom = $userfrom ;   // Now, go through and send messages to other users. for ( $k = 3; $k <= 7; $k ++) { // Don't send to the same user. if ( $i == $k ) { continue ; }   $userto = $DB ->get_record( 'user' , array ( 'id' => $k ));   // Send 4 messages. for ( $z = 1; $z <= 4; $z ++) { $eventdata ->userto = $userto ; $eventdata ->subject = 'This is test subject ' . $z ; $eventdata ->smallmessage = 'This is test ' . $z . ' from ' . $userfrom ->id . ' to ' . $userto ->id; $eventdata ->fullmessage = 'This is test fullmessage ' . $z ; $eventdata ->fullmessagehtml = '<p>' . $eventdata ->fullmessage . '</p>' ; $messageid = message_send( $eventdata );   // Mark half the messages as read. if ( $z % 2 === 0) { $message = $DB ->get_record( 'message' , [ 'id' => $messageid ]); message_mark_message_read( $message , time()); } } } } Log in as each user from s1 to s5. Confirm you have 4 conversations (number next to the messaging icon). Confirm you have 2 unread messages from each user by clicking on the messaging icon (do not click on any of the individual conversations). Check that half the messages are in the 'message_read' table (make note of the ones that are). Once you have repeated the above steps for each user, log in as s5 and delete a message from each conversation (keep track of which messages were deleted). Now, log in as s7 and send a message to s8. Log in as s8, read the message from s7 and send a reply. Log in as s7 and send a reply to s8. Upgrade to integration master. Log in as each user from s1 to s5. Check you see that there are no messages. Go to the messaging index page and confirm there is a warning about the messages being temporarily unavailable. Run the cron. Log in as each user and confirm all is correct. Confirm you have 4 conversations (number next to the messaging icon). Confirm you have 2 unread messages from each user by clicking on the messaging icon (do  not  click on any of the individual conversations). Confirm the 'messages' table is populated with all the messages (there should be 80 messages). Confirm in 'message_user_actions' (action = '1') that half the messages are marked as read + all the messages received by s5 (since they looked at each conversation to delete a message - so should be another half + another 8 read messages in the table). Confirm in 'message_user_actions' (action = '2') that the messages you deleted were deleted. Log in as user s5 and confirm you don't see the messages you deleted before the migration (there will be 1 message missing from each conversation) Confirm the 'message' and 'message_read' tables only contain the 3 messages from s7 and s8 Test 2 Install a fresh site on a checkout of a15c7459360f220a34c138f808340e3848c85b7c (before any new messaging changes landed). Run mdk users and mindev. Use the following script to generate notifications for the users s1 to s5. <?php   define( 'CLI_SCRIPT' , true);   require_once (__DIR__ . '/config.php' ); require_once ( $CFG ->libdir . '/messagelib.php' );   $eventdata = new \core\message\message(); $eventdata ->modulename = 'assign' ; $eventdata ->fullmessage = 'Full message for notification' ; $eventdata ->fullmessageformat = FORMAT_PLAIN; $eventdata ->fullmessagehtml = '<p>Full message HTML for notification</p>' ; $eventdata ->smallmessage = 'This is test notification' ; $eventdata ->name = 'assign_notification' ; $eventdata ->component = 'mod_assign' ; $eventdata ->notification = 1; $eventdata ->contexturl = 'https://www.google.com' ; $eventdata ->contexturlname = 'google' ; $eventdata ->courseid = $SITE ->id;   for ( $i = 3; $i <= 7; $i ++) { // Get the user. $userfrom = $DB ->get_record( 'user' , [ 'id' => $i ]);   // Now, go through and send notifications to other users. for ( $k = 3; $k <= 7; $k ++) { // Don't send to the same user. if ( $i == $k ) { continue ; }   $userto = $DB ->get_record( 'user' , array ( 'id' => $k ));   // Set their preference to receive assign_notifications. set_user_preference( 'message_provider_mod_assign_assign_notification_loggedin' , 'popup' , $userto ); set_user_preference( 'message_provider_mod_assign_assign_notification_loggedoff' , 'popup' , $userto );   $eventdata ->userfrom = $userfrom ; $eventdata ->userto = $userto ; for ( $z = 1; $z <= 4; $z ++) { $eventdata ->subject = 'This is test notification ' . $z . ' from ' . $userfrom ->id . ' to ' . $userto ->id; $messageid = message_send( $eventdata );     // Mark half the messages as read. if ( $z % 2 === 0) { $message = $DB ->get_record( 'message' , [ 'id' => $messageid ]); message_mark_message_read( $message , time()); } } } } Log in as each user from s1 to s5. Confirm you have 16 notifications, 8 of which are read. Check that half the messages are in the 'message_read' table. Upgrade to integration master. Log in as each user from s1 to s5. Check you see that there are no notifications. Go to the notification index page (click 'See all' in notification popover) and confirm there is a warning about the notification being temporarily unavailable. Run the cron. Log in as each user and confirm all is correct. Confirm the 'notifications' table is populated with all the notifications (there should be 80 notifications) and half have the 'timeread' column set. Test 3 This is to be done on integration master. Log in as an Admin. Create a course. Enrol a student. Create an assignment. Log in as the student. Click on ‘Notification preferences’ in the notification popover. Turn on all assignment notifications for the web processor. Submit an assignment. Confirm a notification is created in the notification popover. Test 4 This is to be done on integration master. Log in as an Admin. Create a course. Enrol a student. Create an assignment. Log in as the student. Click on ‘Notification preferences’ in the notification popover. Turn off all assignment notifications for the web processor. Submit an assignment. Confirm no notification is created in the notification popover.
    • Affected Branches:
      MOODLE_35_STABLE
    • Fixed Branches:
      MOODLE_35_STABLE
    • Pull from Repository:
    • Pull Master Branch:
      MDL-61255_master

      Description

      It will be common for sites to have 10 million+ messages/notifications in these tables. We need to create a sensible way to transfer this data to the new table structures.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              markn Mark Nelson
              Reporter:
              markn Mark Nelson
              Peer reviewer:
              Simey Lameze
              Integrator:
              Jake Dallimore
              Tester:
              David Monllaó
              Participants:
              Component watchers:
              Amaia Anabitarte, Carlos Escobedo, Ferran Recio, Sara Arjona (@sarjona)
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:
                Fix Release Date:
                17/May/18