Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-57775

Payload encoding broken by XML-RPC client

    XMLWordPrintable

    Details

    • Testing Instructions:
      Hide
      PHPUnit
      1. Confirm that the tests for the "webservice_xmlrpc_testsuite" are all passing.

      Test 1 - Moodle.net registration (33 and 32 only)

      1. Set up a new site having non-ascii characters in its name
      2. Register the site with moodle.net
      3. When the site is initially registered with a hub, the site name is passed to the hub via the plain HTTP GET request and all encoding is all good. The site should appears with the correct name on https://moodle.net/sites/ list.
      4. Run the scheduled task \core\task\registration_cron_task. It sends the information via the XML-RPC request.
      5. TEST: Make sure that the site name is still correctly displayed on the hub.

      Test 2 - Registering with moodle-local_hub (33 and 32 only)

      1. Install the moodle-local_hub plugin to a test Moodle server.
      2. Comment out the usages of update_sendy_list() here and here (with Sendy not configured, it breaks the registration process).
      3. Configure your local Moodle hub server (https://docs.moodle.org/33/en/Hub_administration#How_to_set_up_a_Moodle_Hub_server)
      4. Register your Moodle instance to the local hub server via "Site administration / Hubs" and entering the local hub's URL and password into the "Private hub URL" and "Password" fields, respectively. Make sure that the site name has non-ASCII characters.
      5. After registering, open your CLI terminal and go to your Moodle instance's root directory and run

        php admin/tool/task/cli/schedule_task.php --execute=\\core\\task\\registration_cron_task
        

      6. Login as admin to your local Moodle hub server.
      7. Go to "Site administration / Hub"
      8. Click "Manage sites"
      9. Click "Search for sites" and check the details of the Moodle instance that you registered with the local hub.
        • Confirm that the Site's name is not garbled.

      Test 3 - MNet keys swap

      1. Attempt to establish an MNet connection between two patched sites
      2. REGRESSION TEST: Make sure that after specifying the other peer's URL, its MNet key is correctly fetched.
      Show
      PHPUnit Confirm that the tests for the " webservice_xmlrpc_testsuite " are all passing. Test 1 - Moodle.net registration (33 and 32 only) Set up a new site having non-ascii characters in its name Register the site with moodle.net When the site is initially registered with a hub, the site name is passed to the hub via the plain HTTP GET request and all encoding is all good. The site should appears with the correct name on https://moodle.net/sites/ list. Run the scheduled task \core\task\registration_cron_task . It sends the information via the XML-RPC request. TEST: Make sure that the site name is still correctly displayed on the hub. Test 2 - Registering with moodle-local_hub (33 and 32 only) Install the moodle-local_hub plugin to a test Moodle server. Comment out the usages of update_sendy_list() here and here (with Sendy not configured, it breaks the registration process). Configure your local Moodle hub server ( https://docs.moodle.org/33/en/Hub_administration#How_to_set_up_a_Moodle_Hub_server ) Register your Moodle instance to the local hub server via " Site administration / Hubs " and entering the local hub's URL and password into the " Private hub URL " and " Password " fields, respectively. Make sure that the site name has non-ASCII characters. After registering, open your CLI terminal and go to your Moodle instance's root directory and run php admin/tool/task/cli/schedule_task.php --execute=\\core\\task\\registration_cron_task Login as admin to your local Moodle hub server. Go to " Site administration / Hub " Click " Manage sites " Click " Search for sites " and check the details of the Moodle instance that you registered with the local hub. Confirm that the Site's name is not garbled. Test 3 - MNet keys swap Attempt to establish an MNet connection between two patched sites REGRESSION TEST: Make sure that after specifying the other peer's URL, its MNet key is correctly fetched.
    • Affected Branches:
      MOODLE_31_STABLE, MOODLE_32_STABLE, MOODLE_33_STABLE
    • Fixed Branches:
      MOODLE_32_STABLE, MOODLE_33_STABLE
    • Pull from Repository:
    • Pull Master Branch:
      MDL-57775-master-xmlrpc
    • Sprint:
      3.4 Sprint 4

      Description

      This core bug has been discovered while I was working on MDLSITE-4726. We had a problem at moodle.net with some sites with non-ascii characters in their names. Their name was displayed wrongly.

      Synopsis (long story short)

      The XML-RPC client in Moodle 3.1 and 3.2 is broken and needs to be fixed. Moodle sites use that client to update their registration information via a scheduled task.

      Description

      As a result of MDL-52209, the Zend framework was removed from XML-RPC web service plugin in Moodle 3.1. Our webservice_xmlrpc_client class - originally extending the Zend_XmlRpc_Client class - was rewritten to use the PHP's xmlrpc extension directly.

      Once a Moodle site is registered with a hub (such as moodle.net), the class webservice_xmlrpc_client is used by the registration_manager::cron() method to periodically update the site info at the hub. This is done by remotely calling the hub's web service function hub_update_site_info().

      As was discovered in MDL-54868, there is a bug in the xmlrpc extension https://bugs.php.net/bug.php?id=41650 that escapes all UTF-8 characters in the RPC request wrongly as individual bytes, which then leads to nonsense result after decoding on the other side. Example:

      xmlrpc_encode_request('method', '€', ['encoding' => 'utf-8'])
      

      produces incorrect:

      <?xml version="1.0" encoding="utf-8"?>
      <methodCall>
      <methodName>method</methodName>
      <params>
       <param>
        <value>
         <string>&#226;&#130;&#172;</string>
        </value>
       </param>
      </params>
      </methodCall>
      

      Such a request, when decoded via xmlrpc_decode($req, 'utf-8') then leads to a rubbish like ⬠instead of .

      Luckily, the fix is trivial. We need to declare additional output option "escaping" for the xmlrpc_encode_request():

      xmlrpc_encode_request('method', '€', ['encoding' => 'utf-8', 'escaping' => 'markup'])
      

      then leads to the correct:

      <?xml version="1.0" encoding="utf-8"?>
      <methodCall>
      <methodName>method</methodName>
      <params>
       <param>
        <value>
         <string>€</string>
        </value>
       </param>
      </params>
      </methodCall>
      

      Unfortunately, in the issue MDL-54868 only the webservice_xmlrpc_server was fixed to have this escaping set correctly. The webservice_xmlrpc_client was forgotten.

      Steps to reproduce

      Set up a new 3.1 or 3.2 site having non-ascii characters in its name. Let us say the name of the site is MOODLE ŠKOLA.

      When a site is initially registered with a hub, the site name is passed to the hub via the plain HTTP GET request and all encoding is all good. The site appears with the correct name on https://moodle.net/sites/ list.

      Once the schedule task core\task\registration_cron_task is executed, it sends the information wrongly encoded/escaped via the XML-RPC request. It will contain something like

      <string>MOODLE &#197;&#160;KOLA</string>
      

      and after decoding, the name of the site appears on moodle.net as MOODLE Å KOLA.

      See MDLSITE-4726 for further examples of what you get from Greek, Russian or Chinese characters.

      p.s. While debugging this, I also realized that moodle.net has another bug so that it still displays a site even after it has been unregistered. You may take this into account before attempting to use moodle.net for testing this.

        Attachments

        1. s1.png
          s1.png
          6 kB
        2. s0.png
          s0.png
          9 kB

          Issue Links

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:
                  Fix Release Date:
                  11/Sep/17