Moodle
  1. Moodle
  2. MDL-37879

Import backup course from 1.9.x to 2.3.4+ course no longer works

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 2.3.4, 2.4.1
    • Fix Version/s: STABLE backlog
    • Component/s: Backup, Course
    • Labels:
    • Affected Branches:
      MOODLE_23_STABLE, MOODLE_24_STABLE
    • Rank:
      47632

      Description

      Here's the Steps:

      1. Backup the course in 1.9.
      2. Go to the course in 2.3.
      3. Select Restore from the Settings menu.
      4. Add the .zip backup file exported from 1.9 using the file uploader and click Restore.
      5. Receive the "not a standard Moodle file" message and click Continue.
      6. Under "Restore into this course," choose "Merge the backup into this course" and click Continue.
      7. Get the "error/file_invalid_path" message

      PHP Error Message:

      [05-Feb-2013 17:55:00 UTC] Default exception handler: error/file_invalid_path Debug:
      Error code: file_invalid_path
      $a contents: C:\moodledata/temp/backup/5e56cabee22590231f5520ba8e44f73d/course_files\Bentley_-_Culture_and_Kultur.pdf

      • line 1215 of \backup\converter\moodle1\lib.php: moodle1_convert_exception thrown
      • line 1302 of \backup\converter\moodle1\lib.php: call to moodle1_file_manager->migrate_file()
      • line 578 of \backup\converter\moodle1\handlerlib.php: call to moodle1_file_manager->migrate_directory()
      • line 566 of \backup\converter\moodle1\handlerlib.php: call to moodle1_files_handler->migrate_course_files()
      • line 312 of \backup\converter\moodle1\handlerlib.php: call to moodle1_files_handler->process()
      • line 371 of \backup\converter\moodle1\lib.php: call to moodle1_root_handler->on_root_element_start()
      • line 756 of \backup\converter\moodle1\lib.php: call to moodle1_converter->path_start_reached()
      • line 228 of \backup\util\xml\parser\processors\simplified_parser_processor.class.php: call to moodle1_parser_processor->notify_path_start()
      • line 102 of \backup\util\xml\parser\processors\simplified_parser_processor.class.php: call to simplified_parser_processor->process_pending_startend_notifications()
      • line 92 of \backup\util\xml\parser\processors\progressive_parser_processor.class.php: call to simplified_parser_processor->process_chunk()
      • line 169 of \backup\util\xml\parser\progressive_parser.class.php: call to progressive_parser_processor->receive_chunk()
      • line 212 of \backup\util\xml\parser\progressive_parser.class.php: call to progressive_parser->publish()
      • line ? of unknownfile: call to progressive_parser->start_tag()
      • line 158 of \backup\util\xml\parser\progressive_parser.class.php: call to xml_parse()
      • line 137 of \backup\util\xml\parser\progressive_parser.class.php: call to progressive_parser->parse()
      • line 150 of \backup\converter\moodle1\lib.php: call to progressive_parser->process()
      • line 129 of \backup\converter\convertlib.php: call to moodle1_converter->execute()
      • line 209 of \backup\util\helper\convert_helper.class.php: call to base_converter->convert()
      • line 421 of \backup\controller\restore_controller.class.php: call to convert_helper::to_moodle2_format()
      • line 35 of \backup\restore.php: call to restore_controller->convert()

      [05-Feb-2013 17:55:01 UTC] Potential coding error - existing temptables found when disposing database. Must be dropped!

        Issue Links

          Activity

          Hide
          Cihat Okan ARIKAN added a comment -

          Same issue occured on my 2.4 while importing 1.x backups.. I think I could manage the problem as this way:
          in the file of ...../moodle/backup/converter/moodle1/lib.php
          at the line of 1216
          // BACKUP RESTORE PROBLEM OCCURED BY THIS SECTION!!!
          if ($sourcefullpath !== clean_param($sourcefullpath, PARAM_PATH))

          { throw new moodle1_convert_exception('file_invalid_path', $sourcefull$ }

          /// END OF ERROR SECTION!!!
          If you comment these 3 lines, backup restoration from 1.9 to 2.4 is completed successfully.

          Show
          Cihat Okan ARIKAN added a comment - Same issue occured on my 2.4 while importing 1.x backups.. I think I could manage the problem as this way: in the file of ...../moodle/backup/converter/moodle1/lib.php at the line of 1216 // BACKUP RESTORE PROBLEM OCCURED BY THIS SECTION!!! if ($sourcefullpath !== clean_param($sourcefullpath, PARAM_PATH)) { throw new moodle1_convert_exception('file_invalid_path', $sourcefull$ } /// END OF ERROR SECTION!!! If you comment these 3 lines, backup restoration from 1.9 to 2.4 is completed successfully.
          Hide
          Mary Cooch added a comment -

          I am experiencing this problem too while backing up a 1.9.18 course onto a 2.4 site. I will try out your workaround thanks.

          Show
          Mary Cooch added a comment - I am experiencing this problem too while backing up a 1.9.18 course onto a 2.4 site. I will try out your workaround thanks.
          Hide
          Helen Foster added a comment -

          Just linking to an issue I came across when searching for the error message 'file_invalid_path' - MDL-37774 - which may possibly be the cause of this issue.

          MDL-37774 is due to be fixed very soon; when it is, if someone could upgrade to the latest weekly version of 2.3.4+ or 2.4.1+ and check whether this problem is fixed, that would be very helpful.

          Show
          Helen Foster added a comment - Just linking to an issue I came across when searching for the error message 'file_invalid_path' - MDL-37774 - which may possibly be the cause of this issue. MDL-37774 is due to be fixed very soon; when it is, if someone could upgrade to the latest weekly version of 2.3.4+ or 2.4.1+ and check whether this problem is fixed, that would be very helpful.
          Hide
          Helen Foster added a comment -

          Just adding a link to a discussion about this problem: https://moodle.org/mod/forum/discuss.php?d=221856

          Show
          Helen Foster added a comment - Just adding a link to a discussion about this problem: https://moodle.org/mod/forum/discuss.php?d=221856
          Hide
          Mary Cooch added a comment -

          I will check it out when it is fixed, as it's causing a major problem in our school currently.

          Show
          Mary Cooch added a comment - I will check it out when it is fixed, as it's causing a major problem in our school currently.
          Hide
          Matteo Scaramuccia added a comment - - edited

          Hi All,
          the supposed offending code has been reviewed in MDL-37780: the root of this issue is probably the way the path is built under Windows starting from what written in config.php and then with the usage of DIRECTORY_SEPARATOR.

          HTH,
          Matteo

          Show
          Matteo Scaramuccia added a comment - - edited Hi All, the supposed offending code has been reviewed in MDL-37780 : the root of this issue is probably the way the path is built under Windows starting from what written in config.php and then with the usage of DIRECTORY_SEPARATOR . HTH, Matteo
          Hide
          Mary Cooch added a comment -

          I saw that MDL-37774 was fixed today so I upgraded my local host to today's version but I still got the same error messages as before when trying to restore a course.

          Show
          Mary Cooch added a comment - I saw that MDL-37774 was fixed today so I upgraded my local host to today's version but I still got the same error messages as before when trying to restore a course.
          Hide
          Mary Cooch added a comment -

          Tried with 20130222 and still have same error message. We can't restore our 1.9 courses to 2.4. I shall ask the network manager to try the workaround although we shouldn't have to have a workaround.

          Show
          Mary Cooch added a comment - Tried with 20130222 and still have same error message. We can't restore our 1.9 courses to 2.4. I shall ask the network manager to try the workaround although we shouldn't have to have a workaround.
          Hide
          Mark Williams added a comment -

          Have tried Cihat Okan ARIKAN's solution and it seems to work for us. Will await the fix from Moodle.

          Show
          Mark Williams added a comment - Have tried Cihat Okan ARIKAN's solution and it seems to work for us. Will await the fix from Moodle.
          Hide
          Heather Peake added a comment - - edited

          We have the same issue in a Moodle 2.4.3
          I'm not keen on messing with xml files for fear of making more mess than I'm solving. I also have rather a lot of courses to manipulate.

          Show
          Heather Peake added a comment - - edited We have the same issue in a Moodle 2.4.3 I'm not keen on messing with xml files for fear of making more mess than I'm solving. I also have rather a lot of courses to manipulate.
          Hide
          Mary Cooch added a comment -

          Just adding that Cihat's fix worked for my (former) school Moodle too in the meantime.

          Show
          Mary Cooch added a comment - Just adding that Cihat's fix worked for my (former) school Moodle too in the meantime.
          Hide
          David Mudrak added a comment -

          I think that the commit a629ad3dff36554755796c9fbb790db4a02824c4 which is part of MDL-37780 would fix this. But that was fixed in 2.5 only and was not backported. Can somebody confirm that the offending backup can be restored in 2.5?

          Show
          David Mudrak added a comment - I think that the commit a629ad3dff36554755796c9fbb790db4a02824c4 which is part of MDL-37780 would fix this. But that was fixed in 2.5 only and was not backported. Can somebody confirm that the offending backup can be restored in 2.5?
          Hide
          Guo Han Zhong added a comment -

          i tried Arikan workaround above and got this error below instead :

          Debug info:
          Error code: failed_move_converted_into_place
          $a contents:
          Stack trace:

          line 251 of \backup\converter\convertlib.php: convert_exception thrown
          line 132 of \backup\converter\convertlib.php: call to base_converter->replace_tempdir()
          line 209 of \backup\util\helper\convert_helper.class.php: call to base_converter->convert()
          line 421 of \backup\controller\restore_controller.class.php: call to convert_helper::to_moodle2_format()
          line 35 of \backup\restore.php: call to restore_controller->convert()

          Output buffer: <br /> <b>Warning</b>: rename(C:\inetpub\wwwroot\Moodle\moodledata/temp/backup/41232ac7ca4d9773e9fc3faa8e84d9f6_moodle1_c5d42abbf6ed756cf6188229018e707d,C:\inetpub\wwwroot\Moodle\moodledata/temp/backup/41232ac7ca4d9773e9fc3faa8e84d9f6): Access is denied. (code: 5) in <b>C:\inetpub\wwwroot\Moodle\moodle\backup\converter\convertlib.php</b> on line <b>250</b><br />

          By the way, i am running my Moodle on Windows Server 2008 R2, IIS 7.5, MySQL and etc.

          Show
          Guo Han Zhong added a comment - i tried Arikan workaround above and got this error below instead : Debug info: Error code: failed_move_converted_into_place $a contents: Stack trace: line 251 of \backup\converter\convertlib.php: convert_exception thrown line 132 of \backup\converter\convertlib.php: call to base_converter->replace_tempdir() line 209 of \backup\util\helper\convert_helper.class.php: call to base_converter->convert() line 421 of \backup\controller\restore_controller.class.php: call to convert_helper::to_moodle2_format() line 35 of \backup\restore.php: call to restore_controller->convert() Output buffer: <br /> <b>Warning</b>: rename(C:\inetpub\wwwroot\Moodle\moodledata/temp/backup/41232ac7ca4d9773e9fc3faa8e84d9f6_moodle1_c5d42abbf6ed756cf6188229018e707d,C:\inetpub\wwwroot\Moodle\moodledata/temp/backup/41232ac7ca4d9773e9fc3faa8e84d9f6): Access is denied. (code: 5) in <b>C:\inetpub\wwwroot\Moodle\moodle\backup\converter\convertlib.php</b> on line <b>250</b><br /> By the way, i am running my Moodle on Windows Server 2008 R2, IIS 7.5, MySQL and etc.
          Hide
          Derek Chirnside added a comment -

          For your interest: I have restored ("merge into existing course") a 1.9 course OK into Moodle 2.4.3+ (Build: 20130405)
          No problems. As far as I can tell. If there is still a problem of importing, there just be some subtle aspects to this. Some comination of option choice or other factors.
          I tend to think if this was a serious problem there would be more chatter on this thread.

          -Derek

          Show
          Derek Chirnside added a comment - For your interest: I have restored ("merge into existing course") a 1.9 course OK into Moodle 2.4.3+ (Build: 20130405) No problems. As far as I can tell. If there is still a problem of importing, there just be some subtle aspects to this. Some comination of option choice or other factors. I tend to think if this was a serious problem there would be more chatter on this thread. -Derek
          Hide
          David Aylmer added a comment - - edited

          Just came across this today... I can also confirm the workaround works on windows.

          People have already given good explanations for this, here and on the forums, but some additional information...

          The aforementioned fix:
          // if ($sourcefullpath !== clean_param($sourcefullpath, PARAM_PATH)) {
          // throw new moodle1_convert_exception('file_invalid_path', $sourcefullpath);
          // }

          To be clear, the problem here is comparing $sourcefullpath with the retun value of clean_param on windows.

          The backup process takes $CFG->dataroot (which contains double backslashes) and appends more path information to it... stores it in a local variable.... then compares that local with itself being passed to clean_param, PARAM_PATH which converts "\\" to "/" so it always fails the equivalence test.

          Moodle 2.2.11 20130708
          /backup/converter/moodle1/lib.php
          public function migrate_file(...) {

          $sourcefullpath = $this->basepath.'/'.$sourcepath;

          if ($sourcefullpath !== clean_param($sourcefullpath, PARAM_PATH)) {
          throw new moodle1_convert_exception('file_invalid_path', $sourcefullpath);
          }

          /lib/moodlelib.php
          function clean_param

          case PARAM_PATH:
          // Strip all suspicious characters from file path

          $param = str_replace('\\', '/', $param);

          Note: backslashes as directory separators are considered "suspicious characters" in moodlelib!
          It's OK to replace them with forward slashes, but its not OK to then compare that to the original and expect them to be the same.

          Example sourcefullpath = \public\moodledata/temp/backup/635baceb6313f2e148638dc37df37d8b/course_files\file.txt
          Example clean_param(sourcefullpath, PARAM_PATH)) = /public/moodledata/temp/backup/635baceb6313f2e148638dc37df37d8b/course_files/file.txt

          $this->basepath is always based on $CFG->tempdir and tempdir is set by default to $CFG->dataroot/temp
          $CFG->dataroot (moodledata) is built in config.php during the install with double backslashes on windows machines
          These two variables will never be equal on a windows server
          The exception, file_invalid_path, will always be thrown on a windows install of 2.2.11 when attempting a course restore.

          Show
          David Aylmer added a comment - - edited Just came across this today... I can also confirm the workaround works on windows. People have already given good explanations for this, here and on the forums, but some additional information... The aforementioned fix: // if ($sourcefullpath !== clean_param($sourcefullpath, PARAM_PATH)) { // throw new moodle1_convert_exception('file_invalid_path', $sourcefullpath); // } To be clear, the problem here is comparing $sourcefullpath with the retun value of clean_param on windows. The backup process takes $CFG->dataroot (which contains double backslashes) and appends more path information to it... stores it in a local variable.... then compares that local with itself being passed to clean_param, PARAM_PATH which converts "\\" to "/" so it always fails the equivalence test. Moodle 2.2.11 20130708 /backup/converter/moodle1/lib.php public function migrate_file(...) { $sourcefullpath = $this->basepath.'/'.$sourcepath; if ($sourcefullpath !== clean_param($sourcefullpath, PARAM_PATH)) { throw new moodle1_convert_exception('file_invalid_path', $sourcefullpath); } /lib/moodlelib.php function clean_param case PARAM_PATH: // Strip all suspicious characters from file path $param = str_replace('\\', '/', $param); Note: backslashes as directory separators are considered "suspicious characters" in moodlelib! It's OK to replace them with forward slashes, but its not OK to then compare that to the original and expect them to be the same. Example sourcefullpath = \public\moodledata/temp/backup/635baceb6313f2e148638dc37df37d8b/course_files\file.txt Example clean_param(sourcefullpath, PARAM_PATH)) = /public/moodledata/temp/backup/635baceb6313f2e148638dc37df37d8b/course_files/file.txt $this->basepath is always based on $CFG->tempdir and tempdir is set by default to $CFG->dataroot/temp $CFG->dataroot (moodledata) is built in config.php during the install with double backslashes on windows machines These two variables will never be equal on a windows server The exception, file_invalid_path, will always be thrown on a windows install of 2.2.11 when attempting a course restore.
          Hide
          Jean-Michel Vedrine added a comment -

          Hello,
          This was fixed by MDL-39505 in Moodle 2.3.8 and 2.4.5 so if you are running an older version of Moodle and are experiencing this bug the solution is to upgrade to a version where the bug is fixed.
          Note that the bug was not fixed on the 2.2.x branch as this branch was already out of support when MDL-39505 was fixed, so David Aylmer, if you are running Moodle 2.2.11, you need to upgrade to a more recent version (2.3.9+, 2.4.6+ or better 2.5.2+).

          Show
          Jean-Michel Vedrine added a comment - Hello, This was fixed by MDL-39505 in Moodle 2.3.8 and 2.4.5 so if you are running an older version of Moodle and are experiencing this bug the solution is to upgrade to a version where the bug is fixed. Note that the bug was not fixed on the 2.2.x branch as this branch was already out of support when MDL-39505 was fixed, so David Aylmer, if you are running Moodle 2.2.11, you need to upgrade to a more recent version (2.3.9+, 2.4.6+ or better 2.5.2+).
          Hide
          David Aylmer added a comment -

          Thanks for your post Jean,

          A question then.

          If upgrading a 1.9.5 moodle course to 2.5.2, is it still necessary backup and restore using an intermediary version first? The only reason I have a 2.2.11 site is because of a dozen forums posts suggesting that as a staging point for upgrading courses.

          If 2.2.11 is a mandatory staging point, then it should really be under support.
          Are you saying it is possible (and recommended) to upgrade course backups directly from 1.9.5 to 2.5.2?

          That's news!

          -David

          Show
          David Aylmer added a comment - Thanks for your post Jean, A question then. If upgrading a 1.9.5 moodle course to 2.5.2, is it still necessary backup and restore using an intermediary version first? The only reason I have a 2.2.11 site is because of a dozen forums posts suggesting that as a staging point for upgrading courses. If 2.2.11 is a mandatory staging point, then it should really be under support. Are you saying it is possible (and recommended) to upgrade course backups directly from 1.9.5 to 2.5.2? That's news! -David
          Hide
          Jean-Michel Vedrine added a comment - - edited

          I think you are confusing 2 things

          • to upgrade a Moodle website from 1.9 an intermediary step is needed : 1.9.x -> 2.2.11 -> more recent version
          • backup/restore is a very different thing and no intermediary step is needed : you can restore a 1.9 backup in Moodle 2.5 (or 2.6 when it will be released) with no problem. So I suggest you try to restore your 1.9 backups directly on the Moodle website where they will be used.
            The only case when you need to use the upgrade strategy is if you want to keep users data because no users data is restored when you restore a Moodle 1.9 backup on a Moodle 2.x website, so in that case the only working solution is to make a local copy of Moodle 1.9 website, restore your 1.9 course backup with users data in this website, upgrade this website to Moodle 2.2.11 and do a backup of the upgraded course including users data. This is a slow procedure and should only be attempted if you really need the users data of a Moodle 1.9 course (but I have done it many times ).
          Show
          Jean-Michel Vedrine added a comment - - edited I think you are confusing 2 things to upgrade a Moodle website from 1.9 an intermediary step is needed : 1.9.x -> 2.2.11 -> more recent version backup/restore is a very different thing and no intermediary step is needed : you can restore a 1.9 backup in Moodle 2.5 (or 2.6 when it will be released) with no problem. So I suggest you try to restore your 1.9 backups directly on the Moodle website where they will be used. The only case when you need to use the upgrade strategy is if you want to keep users data because no users data is restored when you restore a Moodle 1.9 backup on a Moodle 2.x website, so in that case the only working solution is to make a local copy of Moodle 1.9 website, restore your 1.9 course backup with users data in this website, upgrade this website to Moodle 2.2.11 and do a backup of the upgraded course including users data. This is a slow procedure and should only be attempted if you really need the users data of a Moodle 1.9 course (but I have done it many times ).
          Hide
          Séverin Terrier added a comment -

          David : Moodle 2.2 is necessary when you want to upgrade your entire Moodle "site" from a previous version!
          If you only want to use backup files (from older version), no need to use an intermediate (old) version : Moodle 2.5 can open and use Moodle 1.9 backup files...

          Show
          Séverin Terrier added a comment - David : Moodle 2.2 is necessary when you want to upgrade your entire Moodle "site" from a previous version! If you only want to use backup files (from older version), no need to use an intermediate (old) version : Moodle 2.5 can open and use Moodle 1.9 backup files...

            People

            • Votes:
              16 Vote for this issue
              Watchers:
              23 Start watching this issue

              Dates

              • Created:
                Updated: