Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-58281

Wrong Content-Length in 'readfile_accel' if file does not exist.

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 3.1.5, 3.7.2
    • Fix Version/s: 3.9
    • Component/s: Files API
    • Labels:
    • Testing Instructions:
      Hide

      Non-Files API

      1. First test non file api files, make a test script:

        <?php
        require('config.php');
        require_once($CFG->libdir . '/filelib.php');
        readfile_accel($CFG->libdir.'/thirdpartylibs.xml', 'text/xml', 1); 
        

      2. Call this using curl:

        curl -sv http://moodle.local/test.php > /dev/null 
        

      3. Note the content length (e.g. 8918)
      4. Test is also works fine with a range request

        curl -sv -H "Range: bytes=0-33" http://moodle.local/test.php
        

        Content-Length: 34

      5. Now change the test to a file path that doesn't exist

        <?php
        require('config.php');
        require_once($CFG->libdir . '/filelib.php');
        readfile_accel($CFG->libdir.'/nope.xml', 'text/xml', 1); 
        

      6. Fetch the file again:

        curl -sv [http://moodle.local/test.php]
        

        1. Confirm that there was no Content-Length specified
      7. Fetch the same page in your browser
        1. Confirm that the Moodle exception was shown

      Files API

      1. Log in to Moodle as admin
      2. View the Site home
      3. Turn editing on
      4. Create a new label and upload a PNG image to it
      5. Save changes
      6. Run the following DB query in your preferred DB client:

        select * from mdl_files where filearea = 'mod_label' order by id desc limit 2;
        

      7. Note the contenthash, and the filesize for later, e.g.

        4e1a025e5a09cba870cfa394faa8270524170f1e
        

      8. Right click on the image, and copy the image address
      9. Fetch it using curl:

        curl -sv [url] > /dev/null
        

        1. Confirm that the Content-Length matches the value you noted earlier
      10. Now fetch just the first 32 bytes:

        curl -sv -H "Range: bytes=0-31" [url] --output -
        

        ## Confirm that the new Content-Length is 32

        1. Confirm that the output started with the following

          �PNG
          

      11. Find the file in your moodledata directory using the contenthash from earlier
        For example, if the hash was 4e1a025e5a09cba870cfa394faa8270524170f1e you'll find it in

        /[datadir]/4e/1a/4e1a025e5a09cba870cfa394faa8270524170f1e
        

      12. Rename the file by adding an extension to it:

        /[datadir]/4e/1a/4e1a025e5a09cba870cfa394faa8270524170f1e.bak
        

      13. Open the URL in your browser again and force refresh of the page
        1. Confirm that you were shown an error that that the file could not be found and
        2. Confirm that the error message included the Error code: storedfilecannotreadfile
      14. Perform a range request again (same as before)

        curl -sv -H "Range: bytes=0-31" [url] --output -
        

        1. Confirm that there was no Content-Length
        2. Confirm that a load of HTML was output and not just the first 32 bytes
      Show
      Non-Files API First test non file api files, make a test script: <?php require ( 'config.php' ); require_once ( $CFG ->libdir . '/filelib.php' ); readfile_accel( $CFG ->libdir. '/thirdpartylibs.xml' , 'text/xml' , 1); Call this using curl: curl -sv http: //moodle . local /test .php > /dev/null   Note the content length (e.g. 8918) Test is also works fine with a range request curl -sv -H "Range: bytes=0-33" http: //moodle . local /test .php Content-Length: 34 Now change the test to a file path that doesn't exist <?php require ( 'config.php' ); require_once ( $CFG ->libdir . '/filelib.php' ); readfile_accel( $CFG ->libdir. '/nope.xml' , 'text/xml' , 1); Fetch the file again: curl -sv [http: //moodle . local /test .php] Confirm that there was no Content-Length specified Fetch the same page in your browser Confirm that the Moodle exception was shown Files API Log in to Moodle as admin View the Site home Turn editing on Create a new label and upload a PNG image to it Save changes Run the following DB query in your preferred DB client: select * from mdl_files where filearea = 'mod_label' order by id desc limit 2; Note the contenthash , and the filesize for later, e.g. 4e1a025e5a09cba870cfa394faa8270524170f1e Right click on the image, and copy the image address Fetch it using curl: curl -sv [url] > /dev/null Confirm that the Content-Length matches the value you noted earlier Now fetch just the first 32 bytes: curl -sv -H "Range: bytes=0-31" [url] --output - ##  Confirm that the new Content-Length is 32 Confirm that the output started with the following �PNG Find the file in your moodledata directory using the contenthash from earlier For example, if the hash was 4e1a025e5a09cba870cfa394faa8270524170f1e you'll find it in /[datadir]/4e/1a/4e1a025e5a09cba870cfa394faa8270524170f1e Rename the file by adding an extension to it: /[datadir]/4e/1a/4e1a025e5a09cba870cfa394faa8270524170f1e.bak Open the URL in your browser again and force refresh of the page Confirm that you were shown an error that that the file could not be found and Confirm that the error message included the Error code: storedfilecannotreadfile Perform a range request again (same as before) curl -sv -H "Range: bytes=0-31" [url] --output - Confirm that there was no Content-Length Confirm that a load of HTML was output and not just the first 32 bytes
    • Affected Branches:
      MOODLE_31_STABLE, MOODLE_37_STABLE
    • Fixed Branches:
      MOODLE_39_STABLE
    • Pull Master Branch:
      MDL-58281-file-missing-500

      Description

      If a file is missing in the sitedata, the readfile_accel will still output the header with the 'Content-Length' even though it cannot output the file.

      This may confuse the load balancer (leading to a timeout) because it is expecting more data, as the error message in most of the times is going to be smaller than the provided content-length.

        Attachments

        1. MDL-58281_proposed.patch
          0.7 kB
        2. Screenshot_1.png
          Screenshot_1.png
          677 kB
        3. Screenshot_2.png
          Screenshot_2.png
          469 kB
        4. Screenshot_3.png
          Screenshot_3.png
          1.55 MB
        5. Screenshot_4.png
          Screenshot_4.png
          420 kB

          Issue Links

            Activity

              People

              Assignee:
              brendanheywood Brendan Heywood
              Reporter:
              roperto Daniel Thee Roperto
              Peer reviewer:
              Matt Porritt
              Integrator:
              Andrew Nicols
              Tester:
              Janelle Barcega
              Participants:
              Component watchers:
              Matteo Scaramuccia, Andrew Nicols, Jun Pataleta, Michael Hawkins, Shamim Rezaie, Simey Lameze
              Votes:
              3 Vote for this issue
              Watchers:
              9 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:
                Fix Release Date:
                15/Jun/20

                  Time Tracking

                  Estimated:
                  Original Estimate - 0 minutes
                  0m
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 3 hours, 30 minutes
                  3h 30m