Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-76170

LTI legacy grade sync fails if member sync occurs before first user launch

XMLWordPrintable

    • MOODLE_311_STABLE, MOODLE_400_STABLE
    • MOODLE_311_STABLE, MOODLE_400_STABLE
    • MDL-76170-master
    • Hide

      Setup (note we're going to install based on weekly/on-demand since we want to test the upgrade):

      1. Create a new site called "Consumer" and install it as you normally would.
      2. Create another site (this will be called "Tool") but don't install it yet.
      3. On the Tool site checkout the latest weekly/on-demand:
        • Master:

          git checkout fa7a55aa5de4 

        • 400:

          git checkout 141e369a0

        • 311:

          git checkout 98fd15d75

      4. Install the tool site now.
      5. In BOTH sites, disable Site admin > HTTP security >  Blocked hosts to allow localhost-to-localhost connections to be made. Just clear the values and save it empty.
      6. Set up the Tool site as follows:
        • Enable the enrol_lti and auth_lti plugins.
        • Enable frame embedding via site admin.
      7.  On the Tool site, create a course with two assignments.
      8. Return to the course homepage and from the "More" menu, publish as LTI tool with the following values set:
        • LTI version: Legacy LTI.
        • Tool to be published: Select the first assignment.
      9. Press "Add method" to save.
      10. Repeat to publish the second assignment.
      11. When you are returned back to the Published tools page, click the Legacy LTI tab.
      12. Copy the cartridge URL + secret from the first published assignment (keep for later).
      13. Copy the registration URL from the second published assignment (keep for later).
      14. Now, on the consumer site go to site admin > manage tools.
      15. Paste the cartridge URL into "Tool URL" and click "Add legacy LTI".
      16. Enter "testconsumer" for the consumer key when prompted, and paste in the secret.
      17. Save, and you'll see an LTI 1.1 preconfigured tool is ready for use.
      18. Edit the tool you just created and:
        1. make sure under the "Services" section, both "Names and role" and "Assignment and grade services" are enabled.
        2. Rename this tool "1.1 tool".
        3. Save.
      19. In the "Tool URL", paste the registration URL and click "Add legacy LTI".
      20. Accept / follow any prompts, and this will create an LTI 2.0 preconfigured tool.
      21. Edit the tool to rename it "2.0 tool".
      22. Now create a course with 5 students s1..s5.
      23. Use "Add an activity or resource" to add an external tool, and select the 1.1 tool.
      24. Add a second external tool to the course, selecting the 2.0 tool.
      25. As the admin user, from the course homepage, open the 1.1 tool (this just gives the tool the basic information to be able to run the sync). You should see the assignment load in the iFrame on the page.
      26. As the admin user, then open the 2.0 tool (this just gives the tool the basic information to be able to run the sync). You should see the assignment load in the iFrame on the page.

      Create some known bad data as a result of member sync running before students launch into the tool

      1. In the root of the tool site, run the following in CLI:

        php admin/cli/scheduled_task.php --execute="\enrol_lti\task\sync_members"
        

      Confirm grade sync won't work for users synced before launching

      1. Log in to the consumer site as the student s1.
      2. Go to the course and launch the 1.1 tool.
      3. Submit something to the assignment.
      4. Now launch the 2.0 tool.
      5. Submit something to the assignment.
      6. Log out.
      7. Log in to the consumer as the admin user and go to the course.
      8. Launch the 1.1 tool and grade the user s1 30/100.
      9. Launch the 2.0 tool and grade the user s1 40/100.
      10. Now, in the tool site web root, run the legacy grade sync:

         php admin/cli/scheduled_task.php --execute="\enrol_lti\task\sync_grades" 

      11. Verify: you see output in the task run that indicates that grade sync failed. E.g.

        Failure - The grade '0.30' for the user 'XX' in the tool 'YY' for the course 'ZZ' failed to send.
        Failure - The grade '0.40' for the user 'XX' in the tool 'YY' for the course 'ZZ' failed to send.
        

      Upgrade the site

      1. Now checkout the current HEAD in the Tool site only:
        • master:

          git checkout master

        • 400:

          git checkout MOODLE_400_STABLE

        • 311:

          git checkout MOODLE_311_STABLE

          Note: if you're not testing this on integration, you'll need to checkout or apply the patch now!

      1. Run the upgrade for the tool site.
      2. Once the site upgrade finishes, run the following in the root of the tool site via CLI:

        php admin/cli/scheduled_task.php --execute="\enrol_lti\task\sync_grades"
        

      3. Verify: you see output in the task run that indicates that grades 30/100 and 40/100 were sent to the consumer site. E.g.

        Success - The grade '0.30' for the user 'XX' in the tool 'YY' for the course 'ZZ' was sent. 
        Success - The grade '0.40' for the user 'XX' in the tool 'YY' for the course 'ZZ' was sent.

        Note: Ignore the ordering and ignore the XX, YY, ZZ  - these values are irrelevant to what's being tested.

      Regression test (launch, submit, grade for a user who isn't affected by upgrade code)

      1. Now, log in to the consumer site as the admin user.
      2. Go to the course and enrol the student s6.
      3. Log out.
      4. In the tool site root, run the following via CLI:

        php admin/cli/scheduled_task.php --execute="\enrol_lti\task\sync_members"
        

      5. Log in as the student s6.
      6. Go to the course and launch the 1.1 tool.
      7. Submit something to the assignment.
      8. Now launch the 2.0 tool.
      9. Submit something to the assignment.
      10. Log out.
      11. Log in to the consumer as the admin user and go to the course.
      12. Launch the 1.1 tool and grade the user s6 50/100.
      13. Launch the 2.0 tool and grade the user s6 60/100.
      14. Now, in the tool site web root, run the legacy grade sync:

         php admin/cli/scheduled_task.php --execute="\enrol_lti\task\sync_grades" 

      15. Verify: you see output in the task run that indicates that grades 50/100 and 60/100 were sent to the consumer site. E.g.

        Success - The grade '0.50' for the user 'XX' in the tool 'YY' for the course 'ZZ' was sent. 
        Success - The grade '0.60' for the user 'XX' in the tool 'YY' for the course 'ZZ' was sent.

        Note: Ignore the ordering and ignore the XX, YY, ZZ  - these values are irrelevant to what's being tested.

      Show
      Setup (note we're going to install based on weekly/on-demand since we want to test the upgrade): Create a new site called "Consumer" and install it as you normally would. Create another site (this will be called "Tool") but don't install it yet. On the Tool site checkout the latest weekly/on-demand: Master: git checkout fa7a55aa5de4 400: git checkout 141e369a0 311: git checkout 98fd15d75 Install the tool site now. In BOTH sites, disable Site admin > HTTP security >  Blocked hosts to allow localhost-to-localhost connections to be made. Just clear the values and save it empty. Set up the Tool site as follows: Enable the enrol_lti and auth_lti plugins. Enable frame embedding via site admin.  On the Tool site, create a course with two assignments. Return to the course homepage and from the "More" menu, publish as LTI tool with the following values set: LTI version: Legacy LTI. Tool to be published: Select the first assignment. Press "Add method" to save. Repeat to publish the second assignment. When you are returned back to the Published tools page, click the Legacy LTI tab. Copy the cartridge URL + secret from the first published assignment (keep for later). Copy the registration URL from the second published assignment (keep for later). Now, on the consumer site go to site admin > manage tools. Paste the cartridge URL into "Tool URL" and click "Add legacy LTI". Enter "testconsumer" for the consumer key when prompted, and paste in the secret. Save, and you'll see an LTI 1.1 preconfigured tool is ready for use. Edit the tool you just created and: make sure under the "Services" section, both "Names and role" and "Assignment and grade services" are enabled. Rename this tool "1.1 tool". Save. In the "Tool URL", paste the registration URL and click "Add legacy LTI". Accept / follow any prompts, and this will create an LTI 2.0 preconfigured tool. Edit the tool to rename it "2.0 tool". Now create a course with 5 students s1..s5. Use "Add an activity or resource" to add an external tool, and select the 1.1 tool. Add a second external tool to the course, selecting the 2.0 tool. As the admin user, from the course homepage, open the 1.1 tool (this just gives the tool the basic information to be able to run the sync). You should see the assignment load in the iFrame on the page. As the admin user, then open the 2.0 tool (this just gives the tool the basic information to be able to run the sync). You should see the assignment load in the iFrame on the page. Create some known bad data as a result of member sync running before students launch into the tool In the root of the tool site, run the following in CLI: php admin/cli/scheduled_task.php --execute="\enrol_lti\task\sync_members" Confirm grade sync won't work for users synced before launching Log in to the consumer site as the student s1. Go to the course and launch the 1.1 tool. Submit something to the assignment. Now launch the 2.0 tool. Submit something to the assignment. Log out. Log in to the consumer as the admin user and go to the course. Launch the 1.1 tool and grade the user s1 30/100. Launch the 2.0 tool and grade the user s1 40/100. Now, in the tool site web root, run the legacy grade sync: php admin/cli/scheduled_task.php --execute= "\enrol_lti\task\sync_grades" Verify : you see output in the task run that indicates that grade sync failed. E.g. Failure - The grade '0.30' for the user 'XX' in the tool 'YY' for the course 'ZZ' failed to send. Failure - The grade '0.40' for the user 'XX' in the tool 'YY' for the course 'ZZ' failed to send. Upgrade the site Now checkout the current HEAD in the Tool site only: master: git checkout master 400: git checkout MOODLE_400_STABLE 311: git checkout MOODLE_311_STABLE Note: if you're not testing this on integration, you'll need to checkout or apply the patch now! Run the upgrade for the tool site. Once the site upgrade finishes, run the following in the root of the tool site via CLI: php admin/cli/scheduled_task.php --execute="\enrol_lti\task\sync_grades" Verify : you see output in the task run that indicates that grades 30/100 and 40/100 were sent to the consumer site. E.g. Success - The grade '0.30' for the user 'XX' in the tool 'YY' for the course 'ZZ' was sent. Success - The grade '0.40' for the user 'XX' in the tool 'YY' for the course 'ZZ' was sent. Note: Ignore the ordering and ignore the XX, YY, ZZ  - these values are irrelevant to what's being tested. Regression test (launch, submit, grade for a user who isn't affected by upgrade code) Now, log in to the consumer site as the admin user. Go to the course and enrol the student s6. Log out. In the tool site root, run the following via CLI: php admin/cli/scheduled_task.php --execute="\enrol_lti\task\sync_members" Log in as the student s6. Go to the course and launch the 1.1 tool. Submit something to the assignment. Now launch the 2.0 tool. Submit something to the assignment. Log out. Log in to the consumer as the admin user and go to the course. Launch the 1.1 tool and grade the user s6 50/100. Launch the 2.0 tool and grade the user s6 60/100. Now, in the tool site web root, run the legacy grade sync: php admin/cli/scheduled_task.php --execute= "\enrol_lti\task\sync_grades" Verify : you see output in the task run that indicates that grades 50/100 and 60/100 were sent to the consumer site. E.g. Success - The grade '0.50' for the user 'XX' in the tool 'YY' for the course 'ZZ' was sent. Success - The grade '0.60' for the user 'XX' in the tool 'YY' for the course 'ZZ' was sent. Note: Ignore the ordering and ignore the XX, YY, ZZ  - these values are irrelevant to what's being tested.
    • 0
    • Team Hedgehog 4.1 pre 1.1, Team Hedgehog Sprint 1.1

      Problem

      Noticed this bug when patching MDL-76113 and it's pretty major.

      To replicate, we essentially need to run a sync members job before the students launch into the tool the first time (this created bad entries which aren't fixed during subsequent launches, and ruins chances of a grade sync ever working):

      1. Install two Moodle sites. Call one Consumer and the other Provider
      2. In BOTH sites, disable Site admin > HTTP security >  Blocked hosts to allow localhost-to-localhost connections to be made. Just clear the values and save it empty.
      3. Set up the Provider site as follows:
        • Enable the enrol_lti and auth_lti plugins
        • Enable frame embedding via site admin
      4. On the provider site, create a course with two assignments and publish them both using legacy LTI
      5. Copy the cartridge URL + secret from the first published assignment (keep for later)
      6. Copy the registration URL from the second published assignment (keep for later)
      7. Now, on the consumer site go to site admin > manage tools
      8. Paste the cartridge URL and click "Add legacy LTI"
      9. Enter "testconsumer" for the consumer key when prompted, and paste in the secret
      10. Save, and you'll see an LTI 1.1 preconfigured tool is ready for use
      11. Name this tool "1.1 tool"
      12. Paste the registration URL and click "Add legacy LTI"
      13. This will have created an LTI 2.0 preconfigured tool now
      14. Name this tool "2.0 tool"
      15. Now create a course with 5 students s1..s5
      16. Add a new external tool instance and select the 1.1 tool
      17. Add a new external tool instance and select the 2.0 tool
      18. Launch the 1.1 tool as admin
      19. Launch the 2.0 tool as admin
      20. Now, in the provider web root, run legacy member sync:

         php admin/cli/scheduled_task.php --execute="\enrol_lti\task\sync_members"

      21. Now, log in to the consumer site as the student s1
      22. Go to the course and launch the 1.1 tool
      23. Submit something to the assignment
      24. Now launch the 2.0 tool
      25. Submit something to the assignment
      26. Log out
      27. Log in to the consumer as the admin user and go to the course
      28. Launch the 1.1 tool and grade the user s1 50/100
      29. Launch the 2.0 tool and grade the user s1 60/100
      30. Now, in the tool site web root, run the legacy grade sync:

         php admin/cli/scheduled_task.php --execute="\enrol_lti\task\sync_grades" 

        Expected: The grades 50/100 and 60/100 should be sent to the consumer
        Actual: There's a message for both the 50/100 and the 60/100 stating "The grade....xx for the user yy in the tool zz for the course aa failed to send".

      The member sync essentially creates empty enrol_lti_users records, which are never updated properly with the consumer secret, even on subsequent launched by the students. Were they to launch before the task ran, this wouldn't be the case and the secret would be present. The grade sync needs the secret to sign, and therefore the process will fail 100% of the time in such situations. As can be seen, this affects both 1.1 and 2.0 consumers. Given the member sync task is enabled by default, this bug is likely causing grade sync fails almost all the time in the legacy provider, rendering that feature essentially useless.

      Solutions

      The solution is to:

      • Data fix the existing bad rows
      • Make sure we do set the secret when missing during a launch. That way, when the members do eventually complete the activity, we'll have the consumer secret ready for use in the grade sync task, and everything will work as expected.

      Again, normally I'd be reluctant to change these legacy services. However, as with the linked issue MDL-76113, this fix should be fairly trivial and will resolve an absolutely huge number of grade sync failures that are likely occurring in those sites still using the legacy provider (and would reduce the forum posts reporting such things too - always very difficult posts to address/resolve).

        1. master_1_MDL-76170.png
          master_1_MDL-76170.png
          78 kB
        2. master_2_MDL-76170.png
          master_2_MDL-76170.png
          80 kB
        3. master_3_MDL-76170.png
          master_3_MDL-76170.png
          77 kB
        4. master_4_MDL-76170.png
          master_4_MDL-76170.png
          78 kB
        5. master_5_MDL-76170.png
          master_5_MDL-76170.png
          86 kB
        6. sync_failed.png
          sync_failed.png
          324 kB
        7. sync_success.png
          sync_success.png
          323 kB
        8. v311_1_MDL-76170.png
          v311_1_MDL-76170.png
          90 kB
        9. v311_2_MDL-76170.png
          v311_2_MDL-76170.png
          91 kB
        10. v311_3_MDL-76170.png
          v311_3_MDL-76170.png
          80 kB
        11. v311_4_MDL-76170.png
          v311_4_MDL-76170.png
          86 kB
        12. v311_5_MDL-76170.png
          v311_5_MDL-76170.png
          91 kB
        13. v400_1_MDL-76170.png
          v400_1_MDL-76170.png
          78 kB
        14. v400_2_MDL-76170.png
          v400_2_MDL-76170.png
          79 kB
        15. v400_3_MDL-76170.png
          v400_3_MDL-76170.png
          82 kB
        16. v400_4_MDL-76170.png
          v400_4_MDL-76170.png
          84 kB
        17. v400_5_MDL-76170.png
          v400_5_MDL-76170.png
          90 kB

            jaked Jake Dallimore
            jaked Jake Dallimore
            Huong Nguyen Huong Nguyen
            Michael Hawkins Michael Hawkins
            John Edward Pedregosa John Edward Pedregosa
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved:

                Estimated:
                Original Estimate - 3 hours Original Estimate - 3 hours
                3h
                Remaining:
                Remaining Estimate - 0 minutes
                0m
                Logged:
                Time Spent - 1 day, 5 hours, 20 minutes
                1d 5h 20m

                  Error rendering 'clockify-timesheets-time-tracking-reports:timer-sidebar'. Please contact your Jira administrators.