Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-65794

Make some webservice calls idempotent over http GET

    XMLWordPrintable

Details

    • MOODLE_38_STABLE
    • MOODLE_38_STABLE
    • MDL-65794_master
    • Hide
      1. Create a Vanilla Moodle site.
      2. Disable the setting cachejs (Administration -> Appearance -> AJAX and Javascript).
      3. Visit the Log in page.
      4. Click on 'Log in as a guest'.
      5. Open your browser console and switch to the network tab.
      6. Click on 'Calendar' in the navigation on the left.
      7. Check that the service-nologin.php requests are using GET as their request method.
      8. Make note of the cachekey in the URL of the request.
      9. Refresh page and confirm the cachekey has not changed.
      10. Purge your Moodle caches, eg. mdk purge
      11. Refresh the page.
      12. Check that the service-nologin.php requests are using GET as their request method.
      13. Check that the cachekey in the request URL has changed.

      Check the localstorage cache vs the browser internal cache:

      1. Enable the setting cachejs (Administration -> Appearance -> AJAX and Javascript).
      2. Load a page, the calendar
      3. Confirm that localstorage cache is populated with templates (Chrome dev tools > Application > Localstorage > domain )
      4. Clear out localstorage (right click on domain and 'clear')
      5. Load the calendar again, confirm that these are loaded from the browser cache - Dev tools > Network - Status 200 with size: (from disk cache)
      6. purge caches
      7. Load again, confirm these are loaded from server not cache (valid size in network tab)
      Show
      Create a Vanilla Moodle site. Disable the setting cachejs (Administration -> Appearance -> AJAX and Javascript). Visit the Log in page. Click on 'Log in as a guest'. Open your browser console and switch to the network tab. Click on 'Calendar' in the navigation on the left. Check that the service-nologin.php requests are using GET as their request method. Make note of the cachekey in the URL of the request. Refresh page and confirm the cachekey has not changed. Purge your Moodle caches, eg. mdk purge Refresh the page. Check that the service-nologin.php requests are using GET as their request method. Check that the cachekey in the request URL has changed. Check the localstorage cache vs the browser internal cache: Enable the setting cachejs (Administration -> Appearance -> AJAX and Javascript). Load a page, the calendar Confirm that localstorage cache is populated with templates (Chrome dev tools > Application > Localstorage > domain ) Clear out localstorage (right click on domain and 'clear') Load the calendar again, confirm that these are loaded from the browser cache - Dev tools > Network - Status 200 with size: (from disk cache) purge caches Load again, confirm these are loaded from server not cache (valid size in network tab)

    Description

      Here is a quick summary of calls to service-nologin.php

       

      $ tail -n 100000 access.log | grep 'service-nologin' | cut --delim=' ' -f 11,12 | sed 's/sesskey=.*&/sesskey=xxx/' | sort | uniq -c | sort -hr
      14722 "POST /lib/ajax/service-nologin.php?sesskey=xxxinfo=core_output_load_template
      1265 "POST /lib/ajax/service-nologin.php?sesskey=xxxinfo=core_output_load_fontawesome_icon_map
      769 "POST /lib/ajax/service-nologin.php?sesskey=xxxinfo=core_get_string,core_get_string
      725 "POST /lib/ajax/service-nologin.php?sesskey=xxxinfo=core_get_string,core_get_string,core_get_string
      457 "POST /lib/ajax/service-nologin.php?sesskey=xxxinfo=core_get_string,core_get_string,core_get_string,core_get_string
      237 "POST /lib/ajax/service-nologin.php?sesskey=xxxinfo=core_get_string,core_get_string,core_get_string,core_get_string,core_get_string
      200 "POST /lib/ajax/service-nologin.php?sesskey=xxxinfo=core_get_string
      9 "POST /lib/ajax/service-nologin.php?sesskey=xxxinfo=25-method-calls
      5 "POST /lib/ajax/service-nologin.php?sesskey=xxxinfo=11-method-calls

      So roughly this means that 15% of the traffic that php is serving is a POST request to something which is not personalised to the user, and is global across the system. Templates are static. Strings are static, The icon maps are static.

      None of this traffic needs to go to php, it could all be cached by varnish / CDN etc. 

      The sesskey in the url is pointless as far as I can see, and both the sesskey and the POST are preventing this from being cached further up the stack.

      Most of these will need a revnumber / cache busting param, but that is already a solved problem and just needs to be applied here too.

      So proposing to allow this endpoint to serve traffic via GET and not require the sesskey. If you try to call it this way with a web service which isn't readonly or requires a user then it should explode.

      Attachments

        1. image-2019-06-20-09-46-55-770.png
          370 kB
          Brendan Heywood
        2. image-2019-07-11-15-03-51-190.png
          119 kB
          Brendan Heywood
        3. MDL-65794.jpg
          94 kB
          Anna Carissa Sadia
        4. MDL-65794 (2).jpg
          80 kB
          Anna Carissa Sadia

        Issue Links

          Activity

            People

              markn Mark Nelson
              brendanheywood Brendan Heywood
              Brendan Heywood Brendan Heywood
              Andrew Lyons Andrew Lyons
              Anna Carissa Sadia Anna Carissa Sadia
              Matteo Scaramuccia, David Woloszyn, Huong Nguyen, Jake Dallimore, Meirza, Michael Hawkins, Raquel Ortega, Safat Shahin, Stevani Andolo, Juan Leyva, David Woloszyn, Huong Nguyen, Jake Dallimore, Meirza, Michael Hawkins, Raquel Ortega, Safat Shahin, Stevani Andolo
              Votes:
              2 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:
                18/Nov/19

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 3 hours, 50 minutes
                  3h 50m