Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-57775

Payload encoding broken by XML-RPC client

    XMLWordPrintable

Details

    • MOODLE_31_STABLE, MOODLE_32_STABLE, MOODLE_33_STABLE
    • MOODLE_32_STABLE, MOODLE_33_STABLE
    • MDL-57775-master-xmlrpc
    • Hide
      PHPUnit
      1. Confirm that the tests for the "webservice_xmlrpc_testsuite" are all passing.

      Test 1 - Moodle.net registration (33 and 32 only)

      1. Set up a new site having non-ascii characters in its name
      2. Register the site with moodle.net
      3. When the site is initially registered with a hub, the site name is passed to the hub via the plain HTTP GET request and all encoding is all good. The site should appears with the correct name on https://moodle.net/sites/ list.
      4. Run the scheduled task \core\task\registration_cron_task. It sends the information via the XML-RPC request.
      5. TEST: Make sure that the site name is still correctly displayed on the hub.

      Test 2 - Registering with moodle-local_hub (33 and 32 only)

      1. Install the moodle-local_hub plugin to a test Moodle server.
      2. Comment out the usages of update_sendy_list() here and here (with Sendy not configured, it breaks the registration process).
      3. Configure your local Moodle hub server (https://docs.moodle.org/33/en/Hub_administration#How_to_set_up_a_Moodle_Hub_server)
      4. Register your Moodle instance to the local hub server via "Site administration / Hubs" and entering the local hub's URL and password into the "Private hub URL" and "Password" fields, respectively. Make sure that the site name has non-ASCII characters.
      5. After registering, open your CLI terminal and go to your Moodle instance's root directory and run

        php admin/tool/task/cli/schedule_task.php --execute=\\core\\task\\registration_cron_task
        

      6. Login as admin to your local Moodle hub server.
      7. Go to "Site administration / Hub"
      8. Click "Manage sites"
      9. Click "Search for sites" and check the details of the Moodle instance that you registered with the local hub.
        • Confirm that the Site's name is not garbled.

      Test 3 - MNet keys swap

      1. Attempt to establish an MNet connection between two patched sites
      2. REGRESSION TEST: Make sure that after specifying the other peer's URL, its MNet key is correctly fetched.
      Show
      PHPUnit Confirm that the tests for the " webservice_xmlrpc_testsuite " are all passing. Test 1 - Moodle.net registration (33 and 32 only) Set up a new site having non-ascii characters in its name Register the site with moodle.net When the site is initially registered with a hub, the site name is passed to the hub via the plain HTTP GET request and all encoding is all good. The site should appears with the correct name on https://moodle.net/sites/ list. Run the scheduled task \core\task\registration_cron_task . It sends the information via the XML-RPC request. TEST: Make sure that the site name is still correctly displayed on the hub. Test 2 - Registering with moodle-local_hub (33 and 32 only) Install the moodle-local_hub plugin to a test Moodle server. Comment out the usages of update_sendy_list() here and here (with Sendy not configured, it breaks the registration process). Configure your local Moodle hub server ( https://docs.moodle.org/33/en/Hub_administration#How_to_set_up_a_Moodle_Hub_server ) Register your Moodle instance to the local hub server via " Site administration / Hubs " and entering the local hub's URL and password into the " Private hub URL " and " Password " fields, respectively. Make sure that the site name has non-ASCII characters. After registering, open your CLI terminal and go to your Moodle instance's root directory and run php admin/tool/task/cli/schedule_task.php --execute=\\core\\task\\registration_cron_task Login as admin to your local Moodle hub server. Go to " Site administration / Hub " Click " Manage sites " Click " Search for sites " and check the details of the Moodle instance that you registered with the local hub. Confirm that the Site's name is not garbled. Test 3 - MNet keys swap Attempt to establish an MNet connection between two patched sites REGRESSION TEST: Make sure that after specifying the other peer's URL, its MNet key is correctly fetched.
    • 3.4 Sprint 4

    Description

      This core bug has been discovered while I was working on MDLSITE-4726. We had a problem at moodle.net with some sites with non-ascii characters in their names. Their name was displayed wrongly.

      Synopsis (long story short)

      The XML-RPC client in Moodle 3.1 and 3.2 is broken and needs to be fixed. Moodle sites use that client to update their registration information via a scheduled task.

      Description

      As a result of MDL-52209, the Zend framework was removed from XML-RPC web service plugin in Moodle 3.1. Our webservice_xmlrpc_client class - originally extending the Zend_XmlRpc_Client class - was rewritten to use the PHP's xmlrpc extension directly.

      Once a Moodle site is registered with a hub (such as moodle.net), the class webservice_xmlrpc_client is used by the registration_manager::cron() method to periodically update the site info at the hub. This is done by remotely calling the hub's web service function hub_update_site_info().

      As was discovered in MDL-54868, there is a bug in the xmlrpc extension https://bugs.php.net/bug.php?id=41650 that escapes all UTF-8 characters in the RPC request wrongly as individual bytes, which then leads to nonsense result after decoding on the other side. Example:

      xmlrpc_encode_request('method', '€', ['encoding' => 'utf-8'])
      

      produces incorrect:

      <?xml version="1.0" encoding="utf-8"?>
      <methodCall>
      <methodName>method</methodName>
      <params>
       <param>
        <value>
         <string>&#226;&#130;&#172;</string>
        </value>
       </param>
      </params>
      </methodCall>
      

      Such a request, when decoded via xmlrpc_decode($req, 'utf-8') then leads to a rubbish like ⬠instead of .

      Luckily, the fix is trivial. We need to declare additional output option "escaping" for the xmlrpc_encode_request():

      xmlrpc_encode_request('method', '€', ['encoding' => 'utf-8', 'escaping' => 'markup'])
      

      then leads to the correct:

      <?xml version="1.0" encoding="utf-8"?>
      <methodCall>
      <methodName>method</methodName>
      <params>
       <param>
        <value>
         <string>€</string>
        </value>
       </param>
      </params>
      </methodCall>
      

      Unfortunately, in the issue MDL-54868 only the webservice_xmlrpc_server was fixed to have this escaping set correctly. The webservice_xmlrpc_client was forgotten.

      Steps to reproduce

      Set up a new 3.1 or 3.2 site having non-ascii characters in its name. Let us say the name of the site is MOODLE ŠKOLA.

      When a site is initially registered with a hub, the site name is passed to the hub via the plain HTTP GET request and all encoding is all good. The site appears with the correct name on https://moodle.net/sites/ list.

      Once the schedule task core\task\registration_cron_task is executed, it sends the information wrongly encoded/escaped via the XML-RPC request. It will contain something like

      <string>MOODLE &#197;&#160;KOLA</string>
      

      and after decoding, the name of the site appears on moodle.net as MOODLE Å KOLA.

      See MDLSITE-4726 for further examples of what you get from Greek, Russian or Chinese characters.

      p.s. While debugging this, I also realized that moodle.net has another bug so that it still displays a site even after it has been unregistered. You may take this into account before attempting to use moodle.net for testing this.

      Attachments

        1. s0.png
          s0.png
          9 kB
        2. s1.png
          s1.png
          6 kB

        Issue Links

          Activity

            People

              mudrd8mz David Mudrák (@mudrd8mz)
              mudrd8mz David Mudrák (@mudrd8mz)
              Simey Lameze Simey Lameze
              Jun Pataleta Jun Pataleta
              Marina Glancy Marina Glancy
              David Woloszyn, Huong Nguyen, Jake Dallimore, Meirza, Michael Hawkins, Raquel Ortega, Safat Shahin, Stevani Andolo, Juan Leyva, David Woloszyn, Huong Nguyen, Jake Dallimore, Meirza, Michael Hawkins, Raquel Ortega, Safat Shahin, Stevani Andolo
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:
                11/Sep/17