Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 2.0
    • Fix Version/s: 2.0
    • Component/s: Backup
    • Labels:
      None
    • Difficulty:
      Difficult
    • Affected Branches:
      MOODLE_20_STABLE
    • Fixed Branches:
      MOODLE_20_STABLE
    • Rank:
      32847

      Description

      Implement restore UI 90% equivalent to the backup one (MDL-22142) but with some differences. Just quick-enumerating them here for further details/discussion.

      A) We need support for date (day/month/year) setting. To be added to course settings, it's missing right now.

      B) A lot of settings become decided by the backup file contents. For example, the root->users setting will be "No" if the backup was generated without users. In that case, it should display a simply label (dependencies of course, will continue being enforced but NOTHING will change that value). I've somehow configured the default values and the list of options of the current select ui elements. But I'm not sure if that's the way. Needs discussion.

      C) Here it's one minimalistic restore API. Follow this steps in order to get it working:

      – Set $CFG->keeptempdirectoriesonbackup = true; in your config.php files
      – Execute one backup if one course containing at least one choice activity.
      – Go to $CFG->dataroot/temp/backup and you will see one xxxxxxx directory. Copy that xxxxxxx
      – Create one script with these contents:

      require_once('config.php');
      require_once($CFG->dirroot . '/backup/util/includes/restore_includes.php');
      
      $tempdir = 'xxxxxxx'; // Put here the name of the temp directory.
      $course_to_restore_to = xx; // Target course where backup will be restored
      $user_doing_the_restore = xx; // ID of the user executing the restore
      
      $rc = new restore_controller($tempdir, $course_to_restore_to, backup::INTERACTIVE_YES, backup::MODE_GENERAL, $user_doing_the_restore
      , backup::TARGET_NEW_COURSE);
      
      if ($rc->get_status() == backup::STATUS_REQUIRE_CONV) {
          echo "format: {$rc->get_format()} detected. Conversion needed";
          $rc->convert();
      }
      
      $rc->finish_ui();
      
      if (!$rc->execute_precheck()) {
          $rc->get_precheck_results();
      } else {
          $rc->execute_plan();
      }
      

      As you can see, API is really similar to the backup one, but have two exceptions, both optional (don't happen always), but cause new request to happen:

      C1 - If the backup format doesn't correspond to one moodle2 format, status will be STATUS_REQUIRE_CONV and conversion must be performed, surely with one page showing the situation (with continue button) and another to show the conversion process.

      C2 - If the execution of the prechecks returns false, that means that something is preventing restore to be executed. The results returned by get_precheck_results() will be an array of warnings and/or errors. warnings will allow to continue with execute_plan() but errors wont. Note there is one new status STATUS_NEED_PRECHECK enforcing the execution of the prechecks.

      Both these C1 and C2 mean that, apart from the "typical" settings pages... we'll need to be able to show (conditionally) other pages (stages) "in the middle" of the restore API. Needs discussion, for sure.

      D - Note we have still pending which information we are going to output while execute_plan() is in action (see, related MDL-22144 and MDL-22143, they apply both to backup and restore).

      E - Tiny detail, once execute_plan() has finished ok, it should show one button to go to the restored course.

      F - I'm sure that there was one F, grrr (and important!). But I've forgotten it. Sure it appears when implementing the whole thing.

      Ciao

        Issue Links

          Activity

          Hide
          Eloy Lafuente (stronk7) added a comment -

          Hi Sam,

          I've just sent to CVS all the code causing restore pre-checks to be executed. Explained, should work like this:

          $rc->execute_precheck() returns true if no errors and no warnings. If any of them, return false.
          $rc->get_precheck_results() returns one array, with 2 elements (optionally), errors and warnings. The former does allow the plan to be executed, the later yes.

          So, code should do something like this:

          if ($rc->execute_precheck()) {
              $rc->execute_plan(); // precheck passed clean, direct execution of plan in same request (execution stage).
          } else {
              $results = $rc->get_precheck_results();
              // Visualization of warnings and errors go here (precheck stage).
              if (empty($results['errors'])) {
                  // Show continue button and allow to execute plan in new request (execution stage)
              } else {
                  // There are errors that prevent plan to be executed, sorry. No continue button at all
              }
          }
          

          Also, there is ONE thing that I'm missing the the restore prechecks and we need to discuss (Martin too). It's how are we going to perform the roles mapping on restore. 100% automatic, skipping non-mapped completely? Allowing the user to do the mapping manually? In any case, note that restore won't be creating new roles ever. Only map/skip are the possibilities.

          Depending of the approach followed... I've to make it one simple precheck (100% automatic), or preload all the user roles, convert them somehow to settings and allow Sam to handle them in some way by the UI. So it is one important thing to discuss.

          Ciao

          PS: Tomorrow I'll be 100% devoted to course/section/activity restore and don't plan to work on the roles thing, but we need that decided ASAP.

          Show
          Eloy Lafuente (stronk7) added a comment - Hi Sam, I've just sent to CVS all the code causing restore pre-checks to be executed. Explained, should work like this: $rc->execute_precheck() returns true if no errors and no warnings. If any of them, return false. $rc->get_precheck_results() returns one array, with 2 elements (optionally), errors and warnings. The former does allow the plan to be executed, the later yes. So, code should do something like this: if ($rc->execute_precheck()) { $rc->execute_plan(); // precheck passed clean, direct execution of plan in same request (execution stage). } else { $results = $rc->get_precheck_results(); // Visualization of warnings and errors go here (precheck stage). if (empty($results['errors'])) { // Show continue button and allow to execute plan in new request (execution stage) } else { // There are errors that prevent plan to be executed, sorry. No continue button at all } } Also, there is ONE thing that I'm missing the the restore prechecks and we need to discuss (Martin too). It's how are we going to perform the roles mapping on restore. 100% automatic, skipping non-mapped completely? Allowing the user to do the mapping manually? In any case, note that restore won't be creating new roles ever. Only map/skip are the possibilities. Depending of the approach followed... I've to make it one simple precheck (100% automatic), or preload all the user roles, convert them somehow to settings and allow Sam to handle them in some way by the UI. So it is one important thing to discuss. Ciao PS: Tomorrow I'll be 100% devoted to course/section/activity restore and don't plan to work on the roles thing, but we need that decided ASAP.
          Hide
          Martin Dougiamas added a comment -

          Yay, thanks Eloy!

          About the roles mapping, I'd ideally like to see:

          1) Roles automatically mapped according to archetype and course settings, and THEN
          2) GUI showing these as defaults, allowing the user to change them if required

          Normally people will click through but it's going to be useful for sites with multiple student roles and so on.

          For 2.0 RC1 it would be OK to drop (2) if we run out of time.

          Show
          Martin Dougiamas added a comment - Yay, thanks Eloy! About the roles mapping, I'd ideally like to see: 1) Roles automatically mapped according to archetype and course settings, and THEN 2) GUI showing these as defaults, allowing the user to change them if required Normally people will click through but it's going to be useful for sites with multiple student roles and so on. For 2.0 RC1 it would be OK to drop (2) if we run out of time.
          Hide
          Eloy Lafuente (stronk7) added a comment -

          roles mapping pre-check in CVS with controller integration. Notes:

          • Once pre-checks have been executed, you can take a look to this object:
            $controller->get_info()->role_mappings;
            

          It will contain one array of objects. The keys are the id of the role in the backup file and the targetroleid within each role is the best mapping that the pre-checks have been able to assign to it, with 0 meaning "no mapping found, that role will be ignored on restore execution". The objects have also other attributes that are the name/shortname/archetype of the role in the backup file.

          If you want to construct one choices menu, you'll need to call get_assignable_roles($coursectx, ROLENAME_SHORT, false, $userid) to see which roles the user is authorized to assign.

          Then, the important bit is that, once your UI for change role mappings have been executed, you must do 2 things:

          1) Update each one of the objects, so their targetroleid will contain the manual mappings.
          2) set the modified flag to true. That way restore execution will detect your changes and apply them as definitive role mappings.

          Important note: the mappings coming from UI aren't re-checked at all, restore will, simply, use them, so it's the responsibility of the UI to allow only correct mappings.

          Another note: I've added one param to the execute_precheck() function:

          function execute_precheck($droptemptablesafter = false)
          

          I guess you'll want to execute it with the param set to true, as far as there are going to be, always, multiple requests after the prechecks. It causes the temp tables (already containing precheck results) to be dropped properly, then your post-pre-checks UI will happen and later, the plan execution will recalculate all that info.

          Ciao

          Show
          Eloy Lafuente (stronk7) added a comment - roles mapping pre-check in CVS with controller integration. Notes: Once pre-checks have been executed, you can take a look to this object: $controller->get_info()->role_mappings; It will contain one array of objects. The keys are the id of the role in the backup file and the targetroleid within each role is the best mapping that the pre-checks have been able to assign to it, with 0 meaning "no mapping found, that role will be ignored on restore execution". The objects have also other attributes that are the name/shortname/archetype of the role in the backup file. If you want to construct one choices menu, you'll need to call get_assignable_roles($coursectx, ROLENAME_SHORT, false, $userid) to see which roles the user is authorized to assign. Then, the important bit is that, once your UI for change role mappings have been executed, you must do 2 things: 1) Update each one of the objects, so their targetroleid will contain the manual mappings. 2) set the modified flag to true. That way restore execution will detect your changes and apply them as definitive role mappings. Important note: the mappings coming from UI aren't re-checked at all, restore will, simply, use them, so it's the responsibility of the UI to allow only correct mappings. Another note: I've added one param to the execute_precheck() function: function execute_precheck($droptemptablesafter = false ) I guess you'll want to execute it with the param set to true, as far as there are going to be, always, multiple requests after the prechecks. It causes the temp tables (already containing precheck results) to be dropped properly, then your post-pre-checks UI will happen and later, the plan execution will recalculate all that info. Ciao
          Hide
          Eloy Lafuente (stronk7) added a comment -

          And another question:

          When restoring you need to be able to specify manually both the short and full name of the new course (if we are restoring to new course) and also de startdate of the target course (from xml if restoring to new course/overwriting course configuration or from DB if restoring to existing course without overwriting).

          The three (course shortname, fullname and startdate) are available in the course/course.xml file. Have you planned anything about this... or they are 100% news for you? In any case... would it be handy to have that information in the controller->get_info() object for setting initial values and so?

          I think we need to talk a bit about these 3 settings... uhm... ciao

          Show
          Eloy Lafuente (stronk7) added a comment - And another question: When restoring you need to be able to specify manually both the short and full name of the new course (if we are restoring to new course) and also de startdate of the target course (from xml if restoring to new course/overwriting course configuration or from DB if restoring to existing course without overwriting). The three (course shortname, fullname and startdate) are available in the course/course.xml file. Have you planned anything about this... or they are 100% news for you? In any case... would it be handy to have that information in the controller->get_info() object for setting initial values and so? I think we need to talk a bit about these 3 settings... uhm... ciao
          Hide
          Eloy Lafuente (stronk7) added a comment -

          FYI, I've added:

          $info->original_course_fullname
          $info->original_course_shortname
          $info->original_course_startdate

          to the controller->get_info(), so they will be available there to be used as needed.

          see you in a bunch of hours...ciao

          Show
          Eloy Lafuente (stronk7) added a comment - FYI, I've added: $info->original_course_fullname $info->original_course_shortname $info->original_course_startdate to the controller->get_info(), so they will be available there to be used as needed. see you in a bunch of hours...ciao
          Hide
          Sam Hemelryk added a comment -

          Hi Eloy,

          Martin asked me to upload my patch as it stands presently here in case you would like to check it out.
          I'm half way through the adding the new settings presently, they've been added but the old independent ones still exist too

          Cheers
          Sam

          Show
          Sam Hemelryk added a comment - Hi Eloy, Martin asked me to upload my patch as it stands presently here in case you would like to check it out. I'm half way through the adding the new settings presently, they've been added but the old independent ones still exist too Cheers Sam
          Hide
          Sam Hemelryk added a comment -

          Hi Eloy,

          I have attached the latest patch here for you too have a quick look at if you like before it gets commit.

          Cheers
          Sam

          Show
          Sam Hemelryk added a comment - Hi Eloy, I have attached the latest patch here for you too have a quick look at if you like before it gets commit. Cheers Sam
          Hide
          Sam Hemelryk added a comment -

          Note to myself.... look into capabilities and make sure we are respecting all required caps

          Thanks Eloy

          Show
          Sam Hemelryk added a comment - Note to myself.... look into capabilities and make sure we are respecting all required caps Thanks Eloy
          Hide
          Eloy Lafuente (stronk7) added a comment -

          Q: While chatting some hours ago, you commented about one problem on restore happening if I uncheck some of the "section_xx_included" settings. I've tried that and restore has worked without any problem here. Do you remember what did you get exactly? Sure I'm missing something...

          A: I've already fixed the other problem (deletion of temp dir on restore using wrong dir).

          TIA and ciao

          Show
          Eloy Lafuente (stronk7) added a comment - Q: While chatting some hours ago, you commented about one problem on restore happening if I uncheck some of the "section_xx_included" settings. I've tried that and restore has worked without any problem here. Do you remember what did you get exactly? Sure I'm missing something... A: I've already fixed the other problem (deletion of temp dir on restore using wrong dir). TIA and ciao
          Hide
          Eloy Lafuente (stronk7) added a comment -

          QA: Ah, forget my previous question. Finally found it! Problem is in backup that is publishing too much info about sections and activities not included. Going to fix that, thanks!

          Show
          Eloy Lafuente (stronk7) added a comment - QA: Ah, forget my previous question. Finally found it! Problem is in backup that is publishing too much info about sections and activities not included. Going to fix that, thanks!
          Hide
          Eloy Lafuente (stronk7) added a comment - - edited

          QA: Fixed, now, only meta-info about included sections/activities is present in moodle_backup.xml and restore_controller()->info

          Thanks and ciao

          Show
          Eloy Lafuente (stronk7) added a comment - - edited QA: Fixed, now, only meta-info about included sections/activities is present in moodle_backup.xml and restore_controller()->info Thanks and ciao
          Hide
          Eloy Lafuente (stronk7) added a comment -

          Hi Sam,

          just detected that, after extracting the $filename in temp area you aren't deleting the original file. IMO it should be deleted immediately after extracting. Just that, easy, I guess. Ciao

          Show
          Eloy Lafuente (stronk7) added a comment - Hi Sam, just detected that, after extracting the $filename in temp area you aren't deleting the original file. IMO it should be deleted immediately after extracting. Just that, easy, I guess. Ciao
          Hide
          Sam Hemelryk added a comment -

          Hi Eloy,

          I'm going to resolve this issue now and start creating new tracker issues for the remaining items or for any further bugs or improvements.
          I've just commit a patch that achieves the following:

          • Category and course selectors are now paginated controls
          • The roles mapping interface is in place now
          • Added basic PHP docs

          The following changes were also commit yesterday:

          • Category/Course selectors now respect capabilities
          • Show/hide settings from the UI is tested and implemented against overwrite_cfg setting
          • Final stage continue button now uses GET rather than POST
          • The temp file is deleted after successful extraction
          • Missing capabilities added.

          Shortly I will create issues for the following tasks:

          • Create AJAX to allow for interactive use of course and category search within restore.
          • Finish phpdocs for backup/restore UI

          I link those issue to this one once created, feel free to jump on and comment or create more if you spot any

          Cheers
          Sam

          Show
          Sam Hemelryk added a comment - Hi Eloy, I'm going to resolve this issue now and start creating new tracker issues for the remaining items or for any further bugs or improvements. I've just commit a patch that achieves the following: Category and course selectors are now paginated controls The roles mapping interface is in place now Added basic PHP docs The following changes were also commit yesterday: Category/Course selectors now respect capabilities Show/hide settings from the UI is tested and implemented against overwrite_cfg setting Final stage continue button now uses GET rather than POST The temp file is deleted after successful extraction Missing capabilities added. Shortly I will create issues for the following tasks: Create AJAX to allow for interactive use of course and category search within restore. Finish phpdocs for backup/restore UI I link those issue to this one once created, feel free to jump on and comment or create more if you spot any Cheers Sam

            People

            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: