Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-40379

pluginfile.php redirects to login page on mobile data

    Details

    • Testing Instructions:
      Hide

      (difficulty: difficult, requires one of those mobile network providers that apply conversions to at least images)

      1. Create some content, images included, in Moodle e.g. lessons to let Moodle serve those content bits;
      2. The content is regularly showed in the browser with the same resolution;
      3. (Optional but alternative to having a mobile network provider like the one described above) Use the Developer Tools of your browser by pressing F12 and look at the network tab to check that each content delivered by Moodle through its helpers (pluginfile.php) contains no-trasform in the Cache-Control headers, at least once per request.
      Show
      (difficulty: difficult, requires one of those mobile network providers that apply conversions to at least images) Create some content, images included, in Moodle e.g. lessons to let Moodle serve those content bits; The content is regularly showed in the browser with the same resolution; (Optional but alternative to having a mobile network provider like the one described above) Use the Developer Tools of your browser by pressing F12 and look at the network tab to check that each content delivered by Moodle through its helpers ( pluginfile.php ) contains no-trasform in the Cache-Control headers, at least once per request.
    • Affected Branches:
      MOODLE_24_STABLE, MOODLE_25_STABLE, MOODLE_26_STABLE, MOODLE_27_STABLE
    • Fixed Branches:
      MOODLE_24_STABLE, MOODLE_25_STABLE, MOODLE_26_STABLE
    • Pull from Repository:
    • Pull 2.6 Branch:
      m26_MDL-40379_Prevent_Some_Mobile_Network_Providers_From_Modifying_Content
    • Pull Master Branch:
      m27_MDL-40379_Prevent_Some_Mobile_Network_Providers_From_Modifying_Content

      Description

      I'm researching a problem with pluginfile.php. I also checked it already on demo.moodle.net so it's no site/server specific issue itself.

      Whilst beeing on wifi or a regular connection or on a desktop computer, the problem does NOT exist. But if I access e.g. a lesson content page, with a picture in it, which was uploaded with TinyMCE, my browser (also tested various browsers on my android phone) displays a question mark sign. If I refresh that page, the picture shows up. This just happens to me on the Vodafone network in Germany. On the T-Mobile network (also in GER) with an iPhone 5 the picture doesnt even show up upon refresh. Navigating to the next page succeeds, but again, no picture.

      I further investigated this with apache logs/tcpdump on the server side and found out, that the user gets redirected with http code 303 to the login page. So, to see that very lesson page, I need to be logged in already, so this cant be the case. Disabling require_course_login in mod/lesson/lib.php the function lesson_pluginfile solves the issues, but this cant obviously be the solution. Somehow, the mobile data proxies of the carriers interfere with Moodle and/or the session.

      With tcpdump, I noticed, there is no session cookie being sent along with the GET-request for pluginfile.php/ID/mod_lesson/PAGEID/filename.jpg. Any other clues about this?

        Gliffy Diagrams

        1. screenshot-1.jpg
          104 kB
        2. screenshot-2.jpg
          148 kB
        3. screenshot-3.jpg
          107 kB
        4. screenshot-4.jpg
          89 kB

          Activity

          Hide
          fzillner Florian Zillner added a comment -

          no picture on first page load

          Show
          fzillner Florian Zillner added a comment - no picture on first page load
          Hide
          fzillner Florian Zillner added a comment -

          upon reload, the picture shows up

          Show
          fzillner Florian Zillner added a comment - upon reload, the picture shows up
          Hide
          matteo Matteo Scaramuccia added a comment -

          Hi Florian,
          require_course_login($course, true, $cm); impose a nice auto-login for Guests.
          A first branch on the logic above towards a redirect is the forcelogin setting, Force users to login: could you check if that setting is disabled in your Moodle instance and if on could you make a test setting it off?

          If the "off test" will fail, can you log into your Moodle instance checking that the authentication is kept while browsing different Moodle pages?

          As you can understand, we should find the exact scenario to allow replicating the issue otherwise it could be difficult to investigate why under some network circumstances, the delivery of an image of a lesson fails just the first time, due to an HTTP 303 redirect (Moodle requiring login).

          Besides, does your issue appear even on desktop browsing the site using your phone connection? This could confirm the link between a particular network (usually browsing is provided by means of a Proxy) and your issue.

          TIA,
          Matteo

          Show
          matteo Matteo Scaramuccia added a comment - Hi Florian, require_course_login($course, true, $cm); impose a nice auto-login for Guests. A first branch on the logic above towards a redirect is the forcelogin setting, Force users to login : could you check if that setting is disabled in your Moodle instance and if on could you make a test setting it off ? If the " off test" will fail, can you log into your Moodle instance checking that the authentication is kept while browsing different Moodle pages? As you can understand, we should find the exact scenario to allow replicating the issue otherwise it could be difficult to investigate why under some network circumstances, the delivery of an image of a lesson fails just the first time, due to an HTTP 303 redirect (Moodle requiring login). Besides, does your issue appear even on desktop browsing the site using your phone connection? This could confirm the link between a particular network (usually browsing is provided by means of a Proxy) and your issue. TIA, Matteo
          Hide
          fzillner Florian Zillner added a comment -

          Hi Matteo,
          I checked that forcelogin setting. It's set to off. If it's useful: guest-settings autologinguests = Off; guestloginbutton = Hide.

          I understand that replicating this might be tricky. So I try to help/describe the problem as good as I can.
          On a desktop computer, the authentication is kept throughout the entire site. No problem so far. The authentication is even kept on mobile connections (as far as simple browsing through courses goes), but just not for files which are requested by pluginfile.php.

          And yes, the problem even appears if I tether my mobile connection to a desktop computer. It's the same experience, the page loads fine, except for the image.

          Show
          fzillner Florian Zillner added a comment - Hi Matteo, I checked that forcelogin setting. It's set to off. If it's useful: guest-settings autologinguests = Off; guestloginbutton = Hide. I understand that replicating this might be tricky. So I try to help/describe the problem as good as I can. On a desktop computer, the authentication is kept throughout the entire site. No problem so far. The authentication is even kept on mobile connections (as far as simple browsing through courses goes), but just not for files which are requested by pluginfile.php. And yes, the problem even appears if I tether my mobile connection to a desktop computer. It's the same experience, the page loads fine, except for the image.
          Hide
          fzillner Florian Zillner added a comment - - edited

          screenshot-3.jpg from Chrome Dev Tools: using a direct connection via wifi on a desktop computer, the browser sends the session cookie, the image gets loaded as expected.

          Show
          fzillner Florian Zillner added a comment - - edited screenshot-3.jpg from Chrome Dev Tools: using a direct connection via wifi on a desktop computer, the browser sends the session cookie, the image gets loaded as expected.
          Hide
          fzillner Florian Zillner added a comment - - edited

          screenshot-4.jpg from Chrome Dev Tools: using a mobile connection via the carrier proxy on a desktop computer, the browser does not send the session cookie, the browser gets redirected to the login page.

          Show
          fzillner Florian Zillner added a comment - - edited screenshot-4.jpg from Chrome Dev Tools: using a mobile connection via the carrier proxy on a desktop computer, the browser does not send the session cookie, the browser gets redirected to the login page.
          Hide
          matteo Matteo Scaramuccia added a comment -

          That's interesting: so 1.1.1.n/bmi could be the root for all.
          I'm just reading around the net about it and someone - http://stackoverflow.com/questions/4113268/how-to-stop-javascript-injection-from-vodafone-proxy - suggest the adoption of Cache-Control: no-transform to avoid stuff to accelerate the connection.

          If you want to test it, please patch your lib/filelib.php as shown below:

          diff --git a/lib/filelib.php b/lib/filelib.php
          index dbbd782..1c35db6 100644
          --- a/lib/filelib.php
          +++ b/lib/filelib.php
          @@ -2126,11 +2126,11 @@ function send_temp_file($path, $filename, $pathisstring=false) {
           
               header('Content-Disposition: attachment; filename="'.$filename.'"');
               if (strpos($CFG->wwwroot, 'https://') === 0) { //https sites - watch out for IE! KB812935 and KB316431
          -        header('Cache-Control: max-age=10');
          +        header('Cache-Control: max-age=10, no-transform');
                   header('Expires: '. gmdate('D, d M Y H:i:s', 0) .' GMT');
                   header('Pragma: ');
               } else { //normal http - prevent caching at all cost
          -        header('Cache-Control: private, must-revalidate, pre-check=0, post-check=0, max-age=0');
          +        header('Cache-Control: private, must-revalidate, pre-check=0, post-check=0, max-age=0, no-transform');
                   header('Expires: '. gmdate('D, d M Y H:i:s', 0) .' GMT');
                   header('Pragma: no-cache');
               }
           

          In the mean time I'll investigate a bit more root reasons and possible solutions.

          TIA,
          Matteo

          Show
          matteo Matteo Scaramuccia added a comment - That's interesting: so 1.1.1.n/bmi could be the root for all. I'm just reading around the net about it and someone - http://stackoverflow.com/questions/4113268/how-to-stop-javascript-injection-from-vodafone-proxy - suggest the adoption of Cache-Control: no-transform to avoid stuff to accelerate the connection. If you want to test it, please patch your lib/filelib.php as shown below: diff --git a/lib/filelib.php b/lib/filelib.php index dbbd782..1c35db6 100644 --- a/lib/filelib.php +++ b/lib/filelib.php @@ -2126,11 +2126,11 @@ function send_temp_file($path, $filename, $pathisstring=false) {   header('Content-Disposition: attachment; filename="'.$filename.'"'); if (strpos($CFG->wwwroot, 'https://') === 0) { //https sites - watch out for IE! KB812935 and KB316431 - header('Cache-Control: max-age=10'); + header('Cache-Control: max-age=10, no-transform'); header('Expires: '. gmdate('D, d M Y H:i:s', 0) .' GMT'); header('Pragma: '); } else { //normal http - prevent caching at all cost - header('Cache-Control: private, must-revalidate, pre-check=0, post-check=0, max-age=0'); + header('Cache-Control: private, must-revalidate, pre-check=0, post-check=0, max-age=0, no-transform'); header('Expires: '. gmdate('D, d M Y H:i:s', 0) .' GMT'); header('Pragma: no-cache'); }   In the mean time I'll investigate a bit more root reasons and possible solutions. TIA, Matteo
          Hide
          matteo Matteo Scaramuccia added a comment -

          Hi Florian,
          any chance for you to test the above no-transform hack?

          Show
          matteo Matteo Scaramuccia added a comment - Hi Florian, any chance for you to test the above no-transform hack?
          Hide
          fzillner Florian Zillner added a comment -

          Hi Matteo,

          thanks for your patience and the help so far. I had no time over the past few weeks and the holiday season allowed me to postpone this issue.

          Today I've done some testing again and found out your solution helps a bit. However, some page elements are still linked via the Proxy-URL 1.1.1.x/MYSITE and some are not. Because we want the whole site to not being transformed or changed in any way I took your advice (no-transform) and applied it to lib/weblib.php in the send_headers function, with the result of a perfectly unchanged and unharmed response...

          As the code comment suggests, caching is in any way undesired, so I thought the additional no-transform complies with this. These are my changes:

          --- weblib.bak.php      2013-09-06 14:00:08.000000000 +0200
          +++ weblib.php  2013-09-06 14:00:15.000000000 +0200
          @@ -1899,12 +1899,12 @@
           
               if ($cacheable) {
                   // Allow caching on "back" (but not on normal clicks)
          -        @header('Cache-Control: private, pre-check=0, post-check=0, max-age=0');
          +        @header('Cache-Control: private, pre-check=0, post-check=0, max-age=0, no-transform');
                   @header('Pragma: no-cache');
                   @header('Expires: ');
               } else {
                   // Do everything we can to always prevent clients and proxies caching
          -        @header('Cache-Control: no-store, no-cache, must-revalidate');
          +        @header('Cache-Control: no-store, no-cache, must-revalidate, no-transform');
                   @header('Cache-Control: post-check=0, pre-check=0', false);
                   @header('Pragma: no-cache');
                   @header('Expires: Mon, 20 Aug 1969 09:23:00 GMT');

          Show
          fzillner Florian Zillner added a comment - Hi Matteo, thanks for your patience and the help so far. I had no time over the past few weeks and the holiday season allowed me to postpone this issue. Today I've done some testing again and found out your solution helps a bit. However, some page elements are still linked via the Proxy-URL 1.1.1.x/MYSITE and some are not. Because we want the whole site to not being transformed or changed in any way I took your advice (no-transform) and applied it to lib/weblib.php in the send_headers function, with the result of a perfectly unchanged and unharmed response... As the code comment suggests, caching is in any way undesired, so I thought the additional no-transform complies with this. These are my changes: --- weblib.bak.php 2013-09-06 14:00:08.000000000 +0200 +++ weblib.php 2013-09-06 14:00:15.000000000 +0200 @@ -1899,12 +1899,12 @@   if ($cacheable) { // Allow caching on "back" (but not on normal clicks) - @header('Cache-Control: private, pre-check=0, post-check=0, max-age=0'); + @header('Cache-Control: private, pre-check=0, post-check=0, max-age=0, no-transform'); @header('Pragma: no-cache'); @header('Expires: '); } else { // Do everything we can to always prevent clients and proxies caching - @header('Cache-Control: no-store, no-cache, must-revalidate'); + @header('Cache-Control: no-store, no-cache, must-revalidate, no-transform'); @header('Cache-Control: post-check=0, pre-check=0', false); @header('Pragma: no-cache'); @header('Expires: Mon, 20 Aug 1969 09:23:00 GMT');
          Hide
          matteo Matteo Scaramuccia added a comment -

          Added the no-transform directive - Refs.: http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.9.5 - in any place where at least an image is supposed to be delivered. Note: at this time it seems that the only media type being the target of conversions is an image media type though there's no limit to what media types could be transformed.
          Reading at http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.9.1 about private confirms that there is no relation between no-trasform - whose aim is to reduce the amount of traffic before saving cache space - and private so no-transform has been added even when private has been already added.

          Show
          matteo Matteo Scaramuccia added a comment - Added the no-transform directive - Refs.: http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.9.5 - in any place where at least an image is supposed to be delivered. Note: at this time it seems that the only media type being the target of conversions is an image media type though there's no limit to what media types could be transformed . Reading at http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.9.1 about private confirms that there is no relation between no-trasform - whose aim is to reduce the amount of traffic before saving cache space - and private so no-transform has been added even when private has been already added.
          Hide
          matteo Matteo Scaramuccia added a comment -

          Hi Petr,
          based on your past work with the private directive, would you like to peer review these PRs about no-transform?

          TIA,
          Matteo

          Show
          matteo Matteo Scaramuccia added a comment - Hi Petr, based on your past work with the private directive, would you like to peer review these PRs about no-transform ? TIA, Matteo
          Hide
          matteo Matteo Scaramuccia added a comment -

          Added a missing no-transform directive in theme/image.php.

          Show
          matteo Matteo Scaramuccia added a comment - Added a missing no-transform directive in theme/image.php .
          Hide
          skodak Petr Skoda added a comment -

          please do not assign me as reviewer to any issue, I do it myself when I know I will review it, thanks

          Show
          skodak Petr Skoda added a comment - please do not assign me as reviewer to any issue, I do it myself when I know I will review it, thanks
          Hide
          skodak Petr Skoda added a comment -

          nice detective work, +1 for this change, thanks for the report and fix

          Show
          skodak Petr Skoda added a comment - nice detective work, +1 for this change, thanks for the report and fix
          Hide
          poltawski Dan Poltawski added a comment -

          Great work guys! I've integrated this to master, 26, 25 & 24

          Show
          poltawski Dan Poltawski added a comment - Great work guys! I've integrated this to master, 26, 25 & 24
          Hide
          poltawski Dan Poltawski added a comment -

          (I find it sort of offensive that a no-transform header even exists )

          Show
          poltawski Dan Poltawski added a comment - (I find it sort of offensive that a no-transform header even exists )
          Hide
          ankit_frenz Ankit Agarwal added a comment -

          Tested the header, 'no-transform' was present.

          cheers

          Show
          ankit_frenz Ankit Agarwal added a comment - Tested the header, 'no-transform' was present. cheers
          Hide
          damyon Damyon Wiese added a comment -

          David built a framework for behat
          At first just to test this and that
          10000+ steps written
          Sounds like we're all smitten
          And David should be smiling at that

          Thanks for reporting, patching, and testing this issue. It has been released upstream along with 64 others today.

          Show
          damyon Damyon Wiese added a comment - David built a framework for behat At first just to test this and that 10000+ steps written Sounds like we're all smitten And David should be smiling at that Thanks for reporting, patching, and testing this issue. It has been released upstream along with 64 others today.
          Hide
          matteo Matteo Scaramuccia added a comment -

          Just for reference, here is a nice explanation about the Mobile ISP image recompression: http://calendar.perfplanet.com/2013/mobile-isp-image-recompression/. Not found at the time of proposing this fix.

          Show
          matteo Matteo Scaramuccia added a comment - Just for reference, here is a nice explanation about the Mobile ISP image recompression : http://calendar.perfplanet.com/2013/mobile-isp-image-recompression/ . Not found at the time of proposing this fix.

            People

            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:
                Fix Release Date:
                13/Jan/14