Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-78510

curl truncates HTTP responses when using a proxy



      I'm not really sure this counts as a regression (especially as the "bug" itself is to do with a documented libcurl "feature" (more on that below)); but it's something that worked prior to MDL-76370 and now doesn't. It has broken at least one plugin: https://github.com/Panopto/Moodle-2.0-plugin-for-Panopto/issues/201

      To replicate:

      On a 3.9 site:

      1. Set up a proxy:

        docker run -d --name squid-container -e TZ=UTC -p 1729:3128 ubuntu/squid:5.2-22.04_beta

      2. Configure Moodle to use the proxy:
        1. Browse to "Site administration" > "Server" > "HTTP"
        2. Set "Proxy host" to "localhost"
        3. Set "Proxy port" to 1729
      3. Place the following test script in your site root and call it curltest.php:

        $curl = new curl();
        $options = [
            'CURLOPT_VERBOSE' => false,
            'CURLOPT_RETURNTRANSFER' => true,
            'CURLOPT_HEADER' => true,
        $response = $curl->post("https://postman-echo.com/post", ['foo' => 'bar'], $options);
        $actualresponse = (isset($curl->info["header_size"])) ? substr($response, $curl->info["header_size"]) : "";
        echo $actualresponse;

      1. Browse to [YOUR_MOODLE]/curltest.php
      2. Note that the response body appears complete
      3. Update the site to 4.0

        git checkout MOODLE_400_STABLE
        php admin/cli/upgrade

      4. Browse to [YOUR_MOODLE]/curltest.php (do it in a new tab so you can compare to the response from before)
      5. Notice that the response is now cut off at the start

      This is happening because $curl->info["header_size"] corresponds to the value of: https://curl.se/libcurl/c/CURLINFO_HEADER_SIZE.html and as the description states

      The total includes the size of any received headers suppressed by CURLOPT_SUPPRESS_CONNECT_HEADERS.

      When using the Squid proxy, an additional connect header is added, but this curlopt suppresses it; however the value of header_size remains unchanged (i.e., it returns the value as if the additional header is not suppressed).

      As the value of header_size is technically behaving correctly, I'm not sure this even counts as a Moodle bug/regression; but I felt it worth reporting in case anyone else comes up against this.

      It may be worth investigating using curl::strip_double_headers to resolve MDL-76370 instead of CURLOPT_SUPPRESS_CONNECT_HEADERS.

            Unassigned Unassigned
            cameron1729 cameron1729
            2 Vote for this issue
            4 Start watching this issue


                Error rendering 'clockify-timesheets-time-tracking-reports:timer-sidebar'. Please contact your Jira administrators.