Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-58281

Wrong Content-Length in 'readfile_accel' if file does not exist.

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Waiting for peer review
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 3.1.5, 3.7.2
    • Fix Version/s: None
    • Component/s: Files API
    • Labels:
    • Testing Instructions:
      Hide

      1) First test non file api files, make a test script:

      <?php
      require('config.php');
      readfile_accel($CFG->libdir.'/thirdpartylibs.xml', 'text/xml', 1); 
      

      2) Test this works fine 

      curl -sv http://moodle.local/test.php > /dev/null 
      

      Also note the content length

      Content-Length: 8477

      3) Test is also works fine with a range request

      curl -sv -H "Range: bytes=0-33" http://moodle.local/test.php
      

      Content-Length: 36

      4) Now hack the test to a file path that doesn't exist

      <?php
      require('config.php');
      readfile_accel($CFG->libdir.'/nope.xml', 'text/xml', 1); 
      

      curl -sv http://moodle.local/test.php

      This should dump the moodle exception page. The Content length should be the length of the error page, and not 0 and not the length of what the file was before.

      5) Test it in the browser too, you should see a proper error page and not a default browser error page or a blank page

      6) Now lets test File API files. Upload something like an image to moodle and grab the url eg:

      http://moodle.local/pluginfile.php/82/mod_label/intro/Workspace%201_999%28004%29.png
      

      Find and note it's file hash on disk, eg:

      4e1a025e5a09cba870cfa394faa8270524170f1e

      Test that it works normally in a browser and displays.

      7) Test that it works using range requests. In the browser dev tools grab this url as a cURL command eg:

      curl 'http://moodle.local/pluginfile.php/82/mod_label/intro/Workspace%201_999%28004%29.png' -H 'Connection: keep-alive' -H 'Pragma: no-cache' -H 'Cache-Control: no-cache' -H 'Upgrade-Insecure-Requests: 1' -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.96 Safari/537.36' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8' -H 'Referer: http://moodle.local/login/index.php' -H 'Accept-Encoding: gzip, deflate' -H 'Accept-Language: en-US,en;q=0.9,de;q=0.8,ko;q=0.7' -H 'Cookie: MDL_SSP_SessID=c3f68139f8215c875f7e918ffa916617; MoodleSession=lmuishj92tdl5j4dfvp7kohgd7; MOODLEID1_=%2596K%2589%25A37mY%251F%25E1%25F8-' --compressed

       

      Now lets grab just the first 32 bytes, note the H "Range: bytes=0-31" and also the -output - | hexdump  at the end: 

      curl 'http://moodle.local/pluginfile.php/82/mod_label/intro/Workspace%201_999%28004%29.png' -H "Range: bytes=0-31" -H 'Connection: keep-alive' -H 'Pragma: no-cache' -H 'Cache-Control: no-cache' -H 'Upgrade-Insecure-Requests: 1' -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.96 Safari/537.36' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8' -H 'Referer: http://moodle.local/login/index.php' -H 'Accept-Encoding: gzip, deflate' -H 'Accept-Language: en-US,en;q=0.9,de;q=0.8,ko;q=0.7' -H 'Cookie: MDL_SSP_SessID=c3f68139f8215c875f7e918ffa916617; MoodleSession=lmuishj92tdl5j4dfvp7kohgd7; MOODLEID1_=%2596K%2589%25A37mY%251F%25E1%25F8-' --compressed --output - | hexdump

        Note that we get 32 bytes:

      0000000 5089 474e 0a0d 0a1a 0000 0d00 4849 5244
      0000010 0000 9006 0000 1a04 0608 0000 4400 3b97
      0000020
      

      8) Now rename the internal store file to add .bak

      /[datadir]/4e/1a/4e1a025e5a09cba870cfa394faa8270524170f1e.bak

      9) In the browser clear cache or turn them off in the dev tools and try to load the file. You should get the proper styled moodle exception page with the error: 

      Cannot read file 'Workspace 1_999(004).png'. Either the file does not exist or there is a permission problem.

      10) Do a range request, note this time we omit the hexdump bit on the end:

      curl 'http://moodle.local/pluginfile.php/82/mod_label/intro/Workspace%201_999%28004%29.png' -H "Range: bytes=0-31" -H 'Connection: keep-alive' -H 'Pragma: no-cache' -H 'Cache-Control: no-cache' -H 'Upgrade-Insecure-Requests: 1' -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.96 Safari/537.36' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8' -H 'Referer: http://moodle.local/login/index.php' -H 'Accept-Encoding: gzip, deflate' -H 'Accept-Language: en-US,en;q=0.9,de;q=0.8,ko;q=0.7' -H 'Cookie: MDL_SSP_SessID=c3f68139f8215c875f7e918ffa916617; MoodleSession=lmuishj92tdl5j4dfvp7kohgd7; MOODLEID1_=%2596K%2589%25A37mY%251F%25E1%25F8-' --compressed --output -

      This will output the full styled error page again with a content length of the error page not the file or range request.

      Show
      1) First test non file api files, make a test script: <?php require ( 'config.php' ); readfile_accel( $CFG ->libdir. '/thirdpartylibs.xml' , 'text/xml' , 1); 2) Test this works fine  curl -sv http: //moodle . local /test .php > /dev/null   Also note the content length Content-Length: 8477 3) Test is also works fine with a range request curl -sv -H "Range: bytes=0-33" http: //moodle . local /test .php Content-Length: 36 4) Now hack the test to a file path that doesn't exist <?php require ( 'config.php' ); readfile_accel( $CFG ->libdir. '/nope.xml' , 'text/xml' , 1); curl -sv http://moodle.local/test.php This should dump the moodle exception page. The Content length should be the length of the error page, and not 0 and not the length of what the file was before. 5) Test it in the browser too, you should see a proper error page and not a default browser error page or a blank page 6) Now lets test File API files. Upload something like an image to moodle and grab the url eg: http: //moodle . local /pluginfile .php /82/mod_label/intro/Workspace %201_999%28004%29.png Find and note it's file hash on disk, eg: 4e1a025e5a09cba870cfa394faa8270524170f1e Test that it works normally in a browser and displays. 7) Test that it works using range requests. In the browser dev tools grab this url as a cURL command eg: curl 'http://moodle.local/pluginfile.php/82/mod_label/intro/Workspace%201_999%28004%29.png' -H 'Connection: keep-alive' -H 'Pragma: no-cache' -H 'Cache-Control: no-cache' -H 'Upgrade-Insecure-Requests: 1' -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.96 Safari/537.36' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8' -H 'Referer: http://moodle.local/login/index.php' -H 'Accept-Encoding: gzip, deflate' -H 'Accept-Language: en-US,en;q=0.9,de;q=0.8,ko;q=0.7' -H 'Cookie: MDL_SSP_SessID=c3f68139f8215c875f7e918ffa916617; MoodleSession=lmuishj92tdl5j4dfvp7kohgd7; MOODLEID1_=%2596K%2589%25A37mY%251F%25E1%25F8-' --compressed   Now lets grab just the first 32 bytes, note the H "Range: bytes=0-31" and also the  -output - | hexdump   at the end:  curl 'http://moodle.local/pluginfile.php/82/mod_label/intro/Workspace%201_999%28004%29.png' -H "Range: bytes=0-31" -H 'Connection: keep-alive' -H 'Pragma: no-cache' -H 'Cache-Control: no-cache' -H 'Upgrade-Insecure-Requests: 1' -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.96 Safari/537.36' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8' -H 'Referer: http://moodle.local/login/index.php' -H 'Accept-Encoding: gzip, deflate' -H 'Accept-Language: en-US,en;q=0.9,de;q=0.8,ko;q=0.7' -H 'Cookie: MDL_SSP_SessID=c3f68139f8215c875f7e918ffa916617; MoodleSession=lmuishj92tdl5j4dfvp7kohgd7; MOODLEID1_=%2596K%2589%25A37mY%251F%25E1%25F8-' --compressed --output - | hexdump   Note that we get 32 bytes: 0000000 5089 474e 0a0d 0a1a 0000 0d00 4849 5244 0000010 0000 9006 0000 1a04 0608 0000 4400 3b97 0000020 8) Now rename the internal store file to add .bak / [datadir] /4e/1a/4e1a025e5a09cba870cfa394faa8270524170f1e.bak 9) In the browser clear cache or turn them off in the dev tools and try to load the file. You should get the proper styled moodle exception page with the error:  Cannot read file 'Workspace 1_999(004).png'. Either the file does not exist or there is a permission problem. 10) Do a range request, note this time we omit the hexdump bit on the end: curl 'http://moodle.local/pluginfile.php/82/mod_label/intro/Workspace%201_999%28004%29.png' -H "Range: bytes=0-31" -H 'Connection: keep-alive' -H 'Pragma: no-cache' -H 'Cache-Control: no-cache' -H 'Upgrade-Insecure-Requests: 1' -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.96 Safari/537.36' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8' -H 'Referer: http://moodle.local/login/index.php' -H 'Accept-Encoding: gzip, deflate' -H 'Accept-Language: en-US,en;q=0.9,de;q=0.8,ko;q=0.7' -H 'Cookie: MDL_SSP_SessID=c3f68139f8215c875f7e918ffa916617; MoodleSession=lmuishj92tdl5j4dfvp7kohgd7; MOODLEID1_=%2596K%2589%25A37mY%251F%25E1%25F8-' --compressed --output - This will output the full styled error page again with a content length of the error page not the file or range request.
    • Affected Branches:
      MOODLE_31_STABLE, MOODLE_37_STABLE
    • Pull 3.6 Branch:
      MDL-58281-file-missing-500-MOODLE_36_STABLE
    • Pull 3.7 Branch:
      MDL-58281-file-missing-500-MOODLE_37_STABLE
    • Pull Master Branch:
      MDL-58281-file-missing-500

      Description

      If a file is missing in the sitedata, the readfile_accel will still output the header with the 'Content-Length' even though it cannot output the file.

      This may confuse the load balancer (leading to a timeout) because it is expecting more data, as the error message in most of the times is going to be smaller than the provided content-length.

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                3 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated: