Moodle
  1. Moodle
  2. MDL-38509

Implement new admin tool for installing add-ons

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: Future Dev
    • Fix Version/s: 2.5
    • Component/s: Administration
    • Labels:
    • Testing Instructions:
      Hide

      Testing difficulty: medium (may require shell access to tweak permissions)

      1. TEST: Make sure that the new "Add-on installer" admin tool is installed and available via Site administration ► Plugins ► Install add-ons.
      2. Clicking the "Install add-ons from Moodle plugins directory" does not fully work yet because the server side of that feature is not implemented yet (see 4. below for work around).
      3. TEST: Get some ZIP package from Moodle Plugins directory (download them manually to your computer) and try to install it via the new tool.
      4. TEST: To test the "remote" installation mode, we need to mimic the redirection from Moodle Plugins repository back to our machine. You can use the following script to generate a request that simulates what Moodle plugins directory will do:

      <?php
      
      $moodleurl = 'http://localhost/moodle25'; // Replace with your 2.5dev URL
      
      $request = base64_encode(json_encode(array(
          'name'      => 'Stamp collection',
          'component' => 'mod_stampcoll',
          'version'   => 2012062201,
      )));
      
      echo $moodleurl.'/admin/index.php?installaddonrequest='.rawurlencode($request).PHP_EOL;
      

      The script will generate URL like:

      http://localhost/moodle25/admin/index.php?installaddonrequest=eyJuYW1lIjoiU3RhbXAgY29sbGVjdGlvbiIsImNvbXBvbmVudCI6Im1vZF9zdGFtcGNvbGwiLCJ2ZXJzaW9uIjoyMDEyMDYyMjAxfQ%3D%3D
      

      that should lead to installing the described add-on version.

      Show
      Testing difficulty: medium (may require shell access to tweak permissions) 1. TEST: Make sure that the new "Add-on installer" admin tool is installed and available via Site administration ► Plugins ► Install add-ons. 2. Clicking the "Install add-ons from Moodle plugins directory" does not fully work yet because the server side of that feature is not implemented yet (see 4. below for work around). 3. TEST: Get some ZIP package from Moodle Plugins directory (download them manually to your computer) and try to install it via the new tool. 4. TEST: To test the "remote" installation mode, we need to mimic the redirection from Moodle Plugins repository back to our machine. You can use the following script to generate a request that simulates what Moodle plugins directory will do: <?php $moodleurl = 'http: //localhost/moodle25'; // Replace with your 2.5dev URL $request = base64_encode(json_encode(array( 'name' => 'Stamp collection', 'component' => 'mod_stampcoll', 'version' => 2012062201, ))); echo $moodleurl.'/admin/index.php?installaddonrequest='.rawurlencode($request).PHP_EOL; The script will generate URL like: http: //localhost/moodle25/admin/index.php?installaddonrequest=eyJuYW1lIjoiU3RhbXAgY29sbGVjdGlvbiIsImNvbXBvbmVudCI6Im1vZF9zdGFtcGNvbGwiLCJ2ZXJzaW9uIjoyMDEyMDYyMjAxfQ%3D%3D that should lead to installing the described add-on version.
    • Fixed Branches:
      MOODLE_25_STABLE
    • Pull from Repository:
    • Pull Master Branch:
      MDL-38509-tool-installaddon
    • Rank:
      48504

      Description

      We will need the admin interface for installing add-ons. The plugins is supposed:

      1. To serve as a departure and arrival point for communication with moodle.org/plugins
      2. To provide a way for installing add-ons from manually uploaded ZIP file.
      3. To perform some ZIP validation checks (similar to what local_plugins does but not to that extend)

      1. remoteconfirm.png
        16 kB
      2. ui-01.png
        74 kB
      3. validator1.png
        28 kB
      4. validator2.png
        22 kB
      5. validator3.png
        45 kB

        Issue Links

          Activity

          Hide
          David Mudrak added a comment -

          Attaching ui-01.png - the tool screenshot. Note how the tool is displayed in the side administration block. To achieve this, MDL-38497 must be integrated.

          Show
          David Mudrak added a comment - Attaching ui-01.png - the tool screenshot. Note how the tool is displayed in the side administration block. To achieve this, MDL-38497 must be integrated.
          Hide
          Michael de Raadt added a comment -

          I'm not sure if this is work in progress or not. Should it be a sub-task of another meta?

          Show
          Michael de Raadt added a comment - I'm not sure if this is work in progress or not. Should it be a sub-task of another meta?
          Hide
          David Mudrak added a comment -

          Sorry, I forgot to triage this. Yes, it is a work in progress. And yes, META would be nice. However, the parent issue has been reported in MDLSITE project so I added this as a blocker to keep the relation.

          Show
          David Mudrak added a comment - Sorry, I forgot to triage this. Yes, it is a work in progress. And yes, META would be nice. However, the parent issue has been reported in MDLSITE project so I added this as a blocker to keep the relation.
          Hide
          David Mudrak added a comment -

          Some screenshots of the in-built ZIP package validator.

          Show
          David Mudrak added a comment - Some screenshots of the in-built ZIP package validator.
          Hide
          David Mudrak added a comment -

          I'm submitting this for integration now. The feature is implemented as a new admin tool plugin, with minimal impacts on the Moodle core.

          You may notice that the plugin installation mode has been added into the mdeploy.php tool, too. It was finally decided not to use this tool for add-ons installation as there is no real reason to do that (On contrary to automatic updates deployment, that can happen in the midddle of the site upgrade, new add-ons can be added only to fully upgraded and configured site) but the code is still there, should we or somebody else need it in the future.

          Show
          David Mudrak added a comment - I'm submitting this for integration now. The feature is implemented as a new admin tool plugin, with minimal impacts on the Moodle core. You may notice that the plugin installation mode has been added into the mdeploy.php tool, too. It was finally decided not to use this tool for add-ons installation as there is no real reason to do that (On contrary to automatic updates deployment, that can happen in the midddle of the site upgrade, new add-ons can be added only to fully upgraded and configured site) but the code is still there, should we or somebody else need it in the future.
          Hide
          Dan Poltawski added a comment -

          Hi David,

          1. If we have the full environment available, I don't see why you've got a custom downloader implementation in tool_installaddon_installer::download_file()? I'd prefer we were using library functions, so that when we ventually get fixes for things like MDL-33955, they are fixed for all. I'm pretty sure using class curl you could add all your ssl verification checks specific for this.
          2. There is trailing whitespace in admin/tool/installaddon/tests/fixtures/versionphp/version1.php.
          3. There is a phpunit failure:
            There was 1 failure:
            
            1) tool_installaddon_validator_test::test_validate_language_pack
            Failed asserting that false is true.
            
            /Users/danp/git/integration/admin/tool/installaddon/tests/validator_test.php:207
            /Users/danp/git/integration/lib/phpunit/classes/basic_testcase.php:64
            
            To re-run:
             vendor/bin/phpunit tool_installaddon_validator_test admin/tool/installaddon/tests/validator_test.php
            

          I had a little play with it. Seems like it works pretty nicely with whats there so far

          Show
          Dan Poltawski added a comment - Hi David, If we have the full environment available, I don't see why you've got a custom downloader implementation in tool_installaddon_installer::download_file()? I'd prefer we were using library functions, so that when we ventually get fixes for things like MDL-33955 , they are fixed for all. I'm pretty sure using class curl you could add all your ssl verification checks specific for this. There is trailing whitespace in admin/tool/installaddon/tests/fixtures/versionphp/version1.php. There is a phpunit failure: There was 1 failure: 1) tool_installaddon_validator_test::test_validate_language_pack Failed asserting that false is true . /Users/danp/git/integration/admin/tool/installaddon/tests/validator_test.php:207 /Users/danp/git/integration/lib/phpunit/classes/basic_testcase.php:64 To re-run: vendor/bin/phpunit tool_installaddon_validator_test admin/tool/installaddon/tests/validator_test.php I had a little play with it. Seems like it works pretty nicely with whats there so far
          Hide
          CiBoT added a comment -

          Moving this reopened issue out from current integration. Please, re-submit it for integration once ready.

          Show
          CiBoT added a comment - Moving this reopened issue out from current integration. Please, re-submit it for integration once ready.
          Hide
          David Mudrak added a comment -

          Hi Dan. Thanks a lot for looking at the patchset.

          re 1. I would, of course, prefer using core methods for downloading the file, too. But IIRC, last time I was trying to use the core curl class for fetching stuff from moodle.org via HTTPS, I was not able to set its options to eventually provide alternative SSL certificate. Most probably I missed some API/way for doing that. Therefore, I re-used the custom cURL wrapper we have in mdeploy.php. If you see it as a blocker, I shall look at it yet.
          re 2. Yes, there is the trailing whitespace. And it is there intentionally.
          re 3. Confirmed, this is my fault. I did not realize that the test is checking for writable target locations without a testable wrapper. Will fix that today.

          Show
          David Mudrak added a comment - Hi Dan. Thanks a lot for looking at the patchset. re 1. I would, of course, prefer using core methods for downloading the file, too. But IIRC, last time I was trying to use the core curl class for fetching stuff from moodle.org via HTTPS, I was not able to set its options to eventually provide alternative SSL certificate. Most probably I missed some API/way for doing that. Therefore, I re-used the custom cURL wrapper we have in mdeploy.php. If you see it as a blocker, I shall look at it yet. re 2. Yes, there is the trailing whitespace. And it is there intentionally. re 3. Confirmed, this is my fault. I did not realize that the test is checking for writable target locations without a testable wrapper. Will fix that today.
          Hide
          Dan Poltawski added a comment -

          Regarding 1. No, I don't think this blocks this. Although it would be nice if we fixed the curl class to support it.

          Show
          Dan Poltawski added a comment - Regarding 1. No, I don't think this blocks this. Although it would be nice if we fixed the curl class to support it.
          Hide
          David Mudrak added a comment -

          OK, so it turned out that I was not aware of the existence of curl::download_one() method that allows you to download a file with amended cURL options - I thought that there is just the download_file_content() function.

          So, those false unit test failures have been fixed and the curl wrapper class usage has been unified in this new tool_installaddon.

          Also, the new API version of the service at download.moodle.org has been already published, so I improved testing instructions to cover that case, too.

          Show
          David Mudrak added a comment - OK, so it turned out that I was not aware of the existence of curl::download_one() method that allows you to download a file with amended cURL options - I thought that there is just the download_file_content() function. So, those false unit test failures have been fixed and the curl wrapper class usage has been unified in this new tool_installaddon. Also, the new API version of the service at download.moodle.org has been already published, so I improved testing instructions to cover that case, too.
          Hide
          Dan Poltawski added a comment -

          Integrated, thanks David!

          Show
          Dan Poltawski added a comment - Integrated, thanks David!
          Hide
          Dan Poltawski added a comment -

          One nitpicking comment when I was playing with it.. I would prefer this string without "Nothing will be installed yet", even though I understand why its there it felt a bit like it could add confusion. Anyway I defer all things like that to helen, it just didn't sound right to me.
          "There is a request to install add-on <strong>{$a->name}</strong> ({$a->component}) version {$a->version} from the Moodle plugins directory to this site. If you continue, the add-on ZIP package will be downloaded for validation. Nothing will be installed yet."

          Show
          Dan Poltawski added a comment - One nitpicking comment when I was playing with it.. I would prefer this string without "Nothing will be installed yet", even though I understand why its there it felt a bit like it could add confusion. Anyway I defer all things like that to helen, it just didn't sound right to me. "There is a request to install add-on <strong>{$a->name}</strong> ({$a->component}) version {$a->version} from the Moodle plugins directory to this site. If you continue, the add-on ZIP package will be downloaded for validation. Nothing will be installed yet."
          Hide
          David Monllaó added a comment -

          It passes, nice feature!!

          2.- As expected I see An unusual error occurred (tried to reach a page that does not exist). message
          3.- All ok with permissions to install the plugins

          Show
          David Monllaó added a comment - It passes, nice feature!! 2.- As expected I see An unusual error occurred (tried to reach a page that does not exist). message 3.- All ok with permissions to install the plugins
          Hide
          Dan Poltawski added a comment -

          Did you remember to call thankDevelopers() for 'this_weeks_work'? Defaulting to PARAM_SHODDY thanking.

          line 1289 of \lib\changes.php: call to debugging()
          line 281 of \lib\are.php: call to moodleform->detectMissingThanks()
          line 202 of \lib\now.php: call to moodleform->_is_poor_form()
          line 73 of \course\upstream.php: call to moodleform->forgetingToThank()

          Show
          Dan Poltawski added a comment - Did you remember to call thankDevelopers() for 'this_weeks_work'? Defaulting to PARAM_SHODDY thanking. line 1289 of \lib\changes.php: call to debugging() line 281 of \lib\are.php: call to moodleform->detectMissingThanks() line 202 of \lib\now.php: call to moodleform->_is_poor_form() line 73 of \course\upstream.php: call to moodleform->forgetingToThank()
          Hide
          Nadav Kavalerchik added a comment -

          +1 for above comment

          Show
          Nadav Kavalerchik added a comment - +1 for above comment
          Hide
          Helen Foster added a comment -

          Removing qa_test_required label as we now have MDLQA-5719 and MDLQA-5720 (copied and added to the 2.5 QA cycle as MDLQA-5721 and MDLQA-5722).

          Show
          Helen Foster added a comment - Removing qa_test_required label as we now have MDLQA-5719 and MDLQA-5720 (copied and added to the 2.5 QA cycle as MDLQA-5721 and MDLQA-5722 ).
          Hide
          Helen Foster added a comment -

          Removing docs_required label as we now have http://docs.moodle.org/25/en/Installing_add-ons and also http://docs.moodle.org/25/en/error/tool_installaddon/err_curl_exec

          Anyone please feel free to amend or add to as necessary e.g. possible errors.

          Show
          Helen Foster added a comment - Removing docs_required label as we now have http://docs.moodle.org/25/en/Installing_add-ons and also http://docs.moodle.org/25/en/error/tool_installaddon/err_curl_exec Anyone please feel free to amend or add to as necessary e.g. possible errors.

            People

            • Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: