Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-76113

Legacy grade sync always fails for LTI 2.0 consumers

XMLWordPrintable

    • MOODLE_311_STABLE, MOODLE_400_STABLE
    • MOODLE_311_STABLE, MOODLE_400_STABLE
    • MDL-76113-master
    • Hide

      Setup (note we're going to install based on weekly/on-demand since we want to test the upgrade):

      1. Create a new site called "Platform" and install it as you normally would
      2. Create another site (this will be called "Tool") but don't install it yet
      3. On the Tool site checkout the latest weekly/on-demand:
        • Master:

          git checkout fa7a55aa5de4 

        • 400:

          git checkout 141e369a0

        • 311:

          git checkout 98fd15d75

      4. Install the tool site now
      5. In BOTH sites, disable Site admin > HTTP security >  Blocked hosts to allow localhost-to-localhost connections to be made. Just clear the values and save it empty.
      6. Set up the tool site as follows:
        • Enable the enrol_lti and auth_lti plugins
        • Enable frame embedding via site admin
      7. Create a course on the tool site called "LTI 2.0 test course"
      8. Create an assignment named "Assignment 1" in the course
      9. Publish the assignment by going to the course admin's "Published as LTI tools" link and selecting the assignment from the dropdown menu
        • Note, for 400 and master, you should use the "Legacy LTI (1.1/2.0)" tab to publish the assignment over 1.1/2.0
      10. Copy the registration URL from the resulting table and store it somewhere for later.
      11. Create another course on the Tool site. Call this one "LTI 1.1 test course"
      12. Create an assignment named "Assignment 2" in the course
      13. Publish the assignment by going to the course admin's "Published as LTI tools" link and selecting the assignment from the dropdown menu
        • Note, for 400 and master, you should use the "Legacy LTI (1.1/2.0)" tab to publish the assignment over 1.1/2.0
      14. Copy the cartridge URL and secret from the resulting table and store it somewhere for later.
      15. Now, sign in to the Platform site
      16. Go to Site admin > Plugins > Activities > External tool > Manage tools
      17. Paste the registration URL you copied earlier into the text field and click "Add Legacy LTI"
      18. Click to continue when prompted
      19. You should now see your tool in the manage tools listing
      20. Rename the tool "LTI 2.0 tool test"
      21. Now, copy the cartridge URL from the text document
      22. Enter the cartridge URL in the text field and press "Legacy LTI"
      23. When prompted, enter the secret (copy and paste from the text document)
      24. Enter "Consumer_test" for the consumer key (this is arbitrary)
      25. Save
      26. You should now see your tool in the manage tools listing
      27. Rename the tool "LTI 1.1 tool test"
      28. Create a course called "Test course 1"
      29. Enrol a student s1
      30. Click to add and activity, selecting "External tool"
      31. Select "LTI 2.0 tool test" from the list of preconfigured tools
      32. Name the activity instance "LTI 2.0 assignment test"
      33. Save and return to course
      34. Create another course called "Test course 2"
      35. Enrol a student s2
      36. Click to add and activity, selecting "External tool"
      37. Select "LTI 1.1 tool test" from the list of preconfigured tools
      38. Name the activity instance "LTI 2.0 assignment test"
      39. Save and return to course
      40. Create another course called "Test course 3"
      41. Enrol a student s3
      42. Click to add and activity, selecting "External tool"
      43. Select "LTI 1.1 tool test" from the list of preconfigured tools
      44. Name the activity instance "Pre-existing LTI 1.1 assignment "
      45. Save and return to course
      46. Click to add and activity, selecting "External tool"
      47. Select "LTI 2.0 tool test" from the list of preconfigured tools
      48. Name the activity instance "Pre-existing LTI 2.0 assignment "
      49. Save and return to course
      50. Now, log out of the Platform site

      Create some broken legacy data

      1. Log in to the Platform course as the user s3
      2. Go to the course "Test course 3"
      3. Click to launch the "Pre-existing LTI 1.1 assignment" tool
      4. Submit something for the assignment
      5. Click to launch the "Pre-existing LTI 2.0 assignment" tool
      6. Submit something for the assignment
      7. Log out
      8. Now log in to the platform site as the admin user
      9. Go the "Test course 3"
      10. Click to launch the "Pre-existing LTI 1.1 assignment" tool
      11. Grade the user s3, granting 66/100
      12. Click to launch the "Pre-existing LTI 2.0 assignment" tool
      13. Grade the user s3, granting 77/100
      14. Log out

      Confirm grade sync fails for 2.0 consumers

      1. In the root of the tool site, run the following via CLI:

        php admin/cli/scheduled_task.php --execute="\enrol_lti\task\sync_grades"
        

      2. Verify the following output is present (LTI 1.1 grade sync works):

        Success - The grade '0.66' for the user 'XX' in the tool 'YY' for the course 'ZZ' was sent.
        

      3. Verify the following output is present (LTI 2.0 grade sync fails):

        *Failed* - The grade '0.77' for the user 'XX' in the tool 'YY' for the course 'ZZ' *failed to send*.
        

      Upgrade, and confirm data-fix works

      1. Now checkout the current HEAD in the Tool site only:
        • master:

          git checkout master

        • 400:

          git checkout MOODLE_400_STABLE

        • 311:

          git checkout MOODLE_311_STABLE

          Note: if you're not running integration, you'll need to checkout or apply the patch now!

      2. Run the upgrade for the tool site
      3. Once the site upgrade finishes, run the following in the root of the tool site via CLI:

        php admin/cli/scheduled_task.php --execute="\enrol_lti\task\sync_grades"
        

      4. Verify the following output is present (LTI 2.0 grade sync now works):

        Success - The grade '0.77' for the user 'XX' in the tool 'YY' for the course 'ZZ' was sent.
        

      Regression testing (submission, grading and grade sync)

      1. Log in to the platform site as the student s1
      2. Go to the course "Test course 1"
      3. Click the "LTI 2.0 assignment test" link to launch the tool
      4. Submit something to the assignment
      5. Now, log out.
      6. Log in to the platform site as the student s2
      7. Go to the course "Test course 2"
      8. Click the "LTI 2.0 assignment test" link to launch the tool
      9. Submit something to the assignment
      10. Now, log out.
      11. Now, log in to the Platform site as the admin again
      12. Go to the course "Test course 1"
      13. Launch the "LTI 2.0 assignment test" activity
      14. Grade the student submission, awarding them 88/100
      15. Go to the course "Test course 2"
      16. Launch the "LTI 1.1 assignment test" activity
      17. Grade the student submission, awarding them 99/100
      18. Now, via the CLI in the root of the Tool site, run:

        php admin/cli/scheduled_task.php --execute="\enrol_lti\task\sync_grades"
        

      19. Verify the following output is present:

        Success - The grade '0.88' for the user 'XX' in the tool 'YY' for the course 'ZZ' was sent.
        

      20. Verify the following output is present:

        Success - The grade '0.99' for the user 'XX' in the tool 'YY' for the course 'ZZ' was sent.
        

        Note: don't worry about the specific IDs for user, tool and course - these will vary depending on your site.

      1. Verify you don't see any output like the following:

        Failed - The grade 'AA' for the user 'XX' in the tool 'YY' for the course 'ZZ' failed to send.
        

      Show
      Setup (note we're going to install based on weekly/on-demand since we want to test the upgrade): Create a new site called "Platform" and install it as you normally would Create another site (this will be called "Tool") but don't install it yet On the Tool site checkout the latest weekly/on-demand: Master: git checkout fa7a55aa5de4 400: git checkout 141e369a0 311: git checkout 98fd15d75 Install the tool site now In BOTH sites, disable Site admin > HTTP security >  Blocked hosts to allow localhost-to-localhost connections to be made. Just clear the values and save it empty. Set up the tool site as follows: Enable the enrol_lti and auth_lti plugins Enable frame embedding via site admin Create a course on the tool site called "LTI 2.0 test course" Create an assignment named "Assignment 1" in the course Publish the assignment by going to the course admin's "Published as LTI tools" link and selecting the assignment from the dropdown menu Note, for 400 and master, you should use the "Legacy LTI (1.1/2.0)" tab to publish the assignment over 1.1/2.0 Copy the registration URL from the resulting table and store it somewhere for later. Create another course on the Tool site. Call this one "LTI 1.1 test course" Create an assignment named "Assignment 2" in the course Publish the assignment by going to the course admin's "Published as LTI tools" link and selecting the assignment from the dropdown menu Note, for 400 and master, you should use the "Legacy LTI (1.1/2.0)" tab to publish the assignment over 1.1/2.0 Copy the cartridge URL and secret from the resulting table and store it somewhere for later. Now, sign in to the Platform site Go to Site admin > Plugins > Activities > External tool > Manage tools Paste the registration URL you copied earlier into the text field and click "Add Legacy LTI" Click to continue when prompted You should now see your tool in the manage tools listing Rename the tool "LTI 2.0 tool test" Now, copy the cartridge URL from the text document Enter the cartridge URL in the text field and press "Legacy LTI" When prompted, enter the secret (copy and paste from the text document) Enter "Consumer_test" for the consumer key (this is arbitrary) Save You should now see your tool in the manage tools listing Rename the tool "LTI 1.1 tool test" Create a course called "Test course 1" Enrol a student s1 Click to add and activity, selecting "External tool" Select "LTI 2.0 tool test" from the list of preconfigured tools Name the activity instance "LTI 2.0 assignment test" Save and return to course Create another course called "Test course 2" Enrol a student s2 Click to add and activity, selecting "External tool" Select "LTI 1.1 tool test" from the list of preconfigured tools Name the activity instance "LTI 2.0 assignment test" Save and return to course Create another course called "Test course 3" Enrol a student s3 Click to add and activity, selecting "External tool" Select "LTI 1.1 tool test" from the list of preconfigured tools Name the activity instance "Pre-existing LTI 1.1 assignment " Save and return to course Click to add and activity, selecting "External tool" Select "LTI 2.0 tool test" from the list of preconfigured tools Name the activity instance "Pre-existing LTI 2.0 assignment " Save and return to course Now, log out of the Platform site Create some broken legacy data Log in to the Platform course as the user s3 Go to the course "Test course 3" Click to launch the "Pre-existing LTI 1.1 assignment" tool Submit something for the assignment Click to launch the "Pre-existing LTI 2.0 assignment" tool Submit something for the assignment Log out Now log in to the platform site as the admin user Go the "Test course 3" Click to launch the "Pre-existing LTI 1.1 assignment" tool Grade the user s3, granting 66/100 Click to launch the "Pre-existing LTI 2.0 assignment" tool Grade the user s3, granting 77/100 Log out Confirm grade sync fails for 2.0 consumers In the root of the tool site, run the following via CLI: php admin/cli/scheduled_task.php --execute="\enrol_lti\task\sync_grades" Verify the following output is present (LTI 1.1 grade sync works): Success - The grade '0.66' for the user 'XX' in the tool 'YY' for the course 'ZZ' was sent. Verify the following output is present (LTI 2.0 grade sync fails): *Failed* - The grade '0.77' for the user 'XX' in the tool 'YY' for the course 'ZZ' *failed to send*. Upgrade, and confirm data-fix works Now checkout the current HEAD in the Tool site only: master: git checkout master 400: git checkout MOODLE_400_STABLE 311: git checkout MOODLE_311_STABLE Note: if you're not running integration, you'll need to checkout or apply the patch now! Run the upgrade for the tool site Once the site upgrade finishes, run the following in the root of the tool site via CLI: php admin/cli/scheduled_task.php --execute="\enrol_lti\task\sync_grades" Verify the following output is present (LTI 2.0 grade sync now works): Success - The grade '0.77' for the user 'XX' in the tool 'YY' for the course 'ZZ' was sent. Regression testing (submission, grading and grade sync) Log in to the platform site as the student s1 Go to the course "Test course 1" Click the "LTI 2.0 assignment test" link to launch the tool Submit something to the assignment Now, log out. Log in to the platform site as the student s2 Go to the course "Test course 2" Click the "LTI 2.0 assignment test" link to launch the tool Submit something to the assignment Now, log out. Now, log in to the Platform site as the admin again Go to the course "Test course 1" Launch the "LTI 2.0 assignment test" activity Grade the student submission, awarding them 88/100 Go to the course "Test course 2" Launch the "LTI 1.1 assignment test" activity Grade the student submission, awarding them 99/100 Now, via the CLI in the root of the Tool site, run: php admin/cli/scheduled_task.php --execute= "\enrol_lti\task\sync_grades" Verify the following output is present: Success - The grade '0.88' for the user 'XX' in the tool 'YY' for the course 'ZZ' was sent. Verify the following output is present: Success - The grade '0.99' for the user 'XX' in the tool 'YY' for the course 'ZZ' was sent. Note: don't worry about the specific IDs for user, tool and course - these will vary depending on your site. Verify you don't see any output like the following: Failed - The grade 'AA' for the user 'XX' in the tool 'YY' for the course 'ZZ' failed to send.
    • 0
    • Team Hedgehog 4.1 pre 1.1, Team Hedgehog Sprint 1.1

      This is like it says on the tin, this feature doesn't work for LTI 2.0 consumers.

      Steps to reproduce

      1. Install two Moodle sites. Call one Platform and the other Tool
      2. In BOTH sites, disable Site admin > HTTP security >  Blocked hosts to allow localhost-to-localhost connections to be made. Just clear the values and save it empty.
      3. Set up the tool site as follows:
        • Enable the enrol_lti and auth_lti plugins
        • Enable frame embedding via site admin
      4. Create a course on the tool site
      5. Create an assignment in the course called "Assignment 1"
      6. Publish the assignment by going to the course admin's "Published as LTI tools" link and selecting the assignment from the dropdown menu
        • Note, for 400 and master, you should use the "Legacy LTI (1.1/2.0)" tab to publish the assignment over 1.1/2.0
      7. Copy the registration URL from the resulting table and store it somewhere for later.
      8. Now, sign in to the Platform site
      9. Go to Site admin > Plugins > Activities > External tool > Manage tools
      10. Paste the registration URL you copied earlier into the text field and click "Add Legacy LTI"
      11. Click to continue when prompted
      12. You should now see your tool in the manage tools listing
      13. Rename the tool "LTI 2.0 tool test"
      14. Create a course
      15. Enrol a student s1
      16. Click to add and activity, selecting "External tool"
      17. Select "LTI 2.0 tool test" from the list of preconfigured tools
      18. Name the activity instance "LTI 2.0 assignment test"
      19. Now, log out of the Platform site
      20. Log in to the platform site as the student s1
      21. Go to the course
      22. Click the "LTI 2.0 assignment test" link to launch the tool
      23. Submit something to the assignment
      24. Now, log out.
      25. Now, log in to the Platform site as the admin again
      26. Go to the course
      27. Launch the "LTI 2.0 assignment test" activity
      28. Grade the student submission, awarding them 88/100
      29. Now, via the CLI in the root of the Tool site, run:

        php admin/cli/scheduled_task.php --execute="\enrol_lti\task\sync_grades"
        

      30. Observe the following output is present:

        Failed - The grade '0.88' for the user 'XX' in the tool 'YY' for the course 'ZZ' failed to send.
        

      Cause

      What's happening is that the legacy sync_grades task fetches the enrol_lti_users records, which contain the wrong consumersecret when the tool has been used with LTI 2.0. This is then used to try to sign the payload, and of course this fails.

      You can see what my database looks like here after a range of 1.0 (1.1) and 2.0 launches, and why the current query is problematic:

      notice in particular the mismatch between ltiuserconsumersecret (the secret in the enrol_lti_users table) and lticonsumersecret (the secret in the enrol_lti_lti2_consumers table). This is only evident in LTI2.0 consumers.

      Solutions

      1. One solution is left joining to the enrol_lti_lti2_consumer table, to check the version of the consumer; if the consumer is LTI 2.0, then we use the secret stored in the enrol_lti_lti2_consumer table, otherwise (if it's 1.1), the secret in the enrol_lti_users table is fine.
      2. Another solution may be to make sure the enrol_lti_user.consumerkey value is correct when creating users over LTI 2.0 launches (and make sure that it also updates correctly for existing users on subsequent launches). This unfortunately means we won't be able to sync grades until all affected users relaunch the tools - which makes it a less appealing solution. This would also involve data-fixing the affected enrol_lti_users records, as we won't catch those with the launch code change.

      Perhaps a combination of the above would also be suitable.

      Generally speaking, I like to leave the legacy code alone these days because those specifications have been deprecated for some time. In this instance, given fix 1 (as above) should be fairly trivial (not to mention LTI 2.0 specific and quite easy to test), I think this is worth fixing. There are probably a good few number of people out there who use the provider in legacy mode and don't realise by using the registration URL, they're using it as LTI 2.0 - meaning they're susceptible to this bug. This then ends in all sorts of grade sync failure reports on the forums, often with the mistaken assumption that the provider is set up using 1.1.

        1. 1.0_vs_2.0_secrets.png
          1.0_vs_2.0_secrets.png
          203 kB
        2. master_3_MDL-76113.png
          master_3_MDL-76113.png
          44 kB
        3. master_4_MDL-76113.png
          master_4_MDL-76113.png
          50 kB
        4. master_5_MDL-76113.png
          master_5_MDL-76113.png
          51 kB
        5. result_1.png
          result_1.png
          267 kB
        6. result_2.png
          result_2.png
          239 kB
        7. result_3.png
          result_3.png
          268 kB
        8. v311_3_MDL-76113.png
          v311_3_MDL-76113.png
          52 kB
        9. v311_4_MDL-76113.png
          v311_4_MDL-76113.png
          58 kB
        10. v311_5_MDL-76113.png
          v311_5_MDL-76113.png
          59 kB
        11. v400_3_MDL-76113.png
          v400_3_MDL-76113.png
          52 kB
        12. v400_4_MDL-76113.png
          v400_4_MDL-76113.png
          57 kB
        13. v400_5_MDL-76113.png
          v400_5_MDL-76113.png
          59 kB

            jaked Jake Dallimore
            jaked Jake Dallimore
            Stevani Andolo Stevani Andolo
            Huong Nguyen Huong Nguyen
            John Edward Pedregosa John Edward Pedregosa
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved:

                Estimated:
                Original Estimate - 3 hours Original Estimate - 3 hours
                3h
                Remaining:
                Remaining Estimate - 0 minutes
                0m
                Logged:
                Time Spent - 1 day, 35 minutes
                1d 35m

                  Error rendering 'clockify-timesheets-time-tracking-reports:timer-sidebar'. Please contact your Jira administrators.