Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-70752

Restore fails when trying to recover a file from remote storage

    XMLWordPrintable

    Details

    • Testing Instructions:
      Hide

      Installation

      1. Configure FTP remote storage as described here https://docs.moodle.org/dev/File_System_API#FTP_file_system or use the following commands.

      • Apply the patch from attachment to create FTP filesystem class:

        git am --whitespace=nowarn < file_system_ftp.patch
        

      • Add the following to your config.php:

        $CFG->alternative_file_system_class = 'core_file_system_ftp';
        $CFG->file_system_ftp_user = 'test';
        $CFG->file_system_ftp_pass = 'test';
        $CFG->file_system_ftp_host = 'localhost';
        $CFG->file_system_ftp_path = 'home/test/testing';
        

       2. Install FTP locally (the following commands are for ubuntu):

      useradd -m test
      passwd test
      apt-get update
      apt-get install vsftpd ufw ftp
      ufw allow 20/tcp
      ufw allow 21/tcp
      

      3. Set write_enable to YES in /etc/vsftpd.conf.

      4. Restart vsftpd:

      service vsftpd restart
      

      5. From the directory that has the file in it connect to FTP: 

      root@ec8b0e0ad064:/siteroot# cd your-dir-with-file
      root@ec8b0e0ad064:/siteroot# ftp localhost
       Connected to localhost.
       220 (vsFTPd 3.0.3)
       Name (localhost:root): test
       331 Please specify the password.
       Password:
       230 Login successful.
       Remote system type is UNIX.
       Using binary mode to transfer files.
      

      Be sure that binary mode is set.
      6. Create root directory for the storage:

      ftp> mkdir /home/test/testing
      ftp> mkdir /home/test/testing/filedir
      ftp> cd /home/test/testing/filedir
      

      7. Upload the file from attachment (b21d09a7fd8a3e5292060b5db4f4f5a412393c24):

      ftp> put b21d09a7fd8a3e5292060b5db4f4f5a412393c24
      

      Test the patch

      1. Access Site administration > Courses > Restore course.
      2. Upload the backup from the attachment (backup-moodle2-course-5-mock_course-20210204-0456-nf.mbz).
      3. Go through the restore pages and perform restore.
      4. Confirm, the course is restored.
      5. Access the course and open Some file resource.
      6. Confirm, the file content is displayed correctly.
      Show
      Installation 1. Configure FTP remote storage as described here https://docs.moodle.org/dev/File_System_API#FTP_file_system  or use the following commands. Apply the patch from attachment to create FTP filesystem class: git am --whitespace=nowarn < file_system_ftp.patch Add the following to your config.php : $CFG->alternative_file_system_class = 'core_file_system_ftp' ; $CFG->file_system_ftp_user = 'test' ; $CFG->file_system_ftp_pass = 'test' ; $CFG->file_system_ftp_host = 'localhost' ; $CFG->file_system_ftp_path = 'home/test/testing' ;  2. Install FTP locally (the following commands are for ubuntu): useradd -m test passwd test apt-get update apt-get install vsftpd ufw ftp ufw allow 20 /tcp ufw allow 21 /tcp 3. Set write_enable to YES in /etc/vsftpd.conf. 4. Restart vsftpd : service vsftpd restart 5. From the directory that has the file in it connect to FTP:  root @ec8b0e0ad064 :/siteroot# cd your-dir-with-file root @ec8b0e0ad064 :/siteroot# ftp localhost Connected to localhost. 220 (vsFTPd 3.0 . 3 ) Name (localhost:root): test 331 Please specify the password. Password: 230 Login successful. Remote system type is UNIX. Using binary mode to transfer files. Be sure that binary mode is set. 6. Create root directory for the storage: ftp> mkdir /home/test/testing ftp> mkdir /home/test/testing/filedir ftp> cd /home/test/testing/filedir 7. Upload the file from attachment ( b21d09a7fd8a3e5292060b5db4f4f5a412393c24 ): ftp> put b21d09a7fd8a3e5292060b5db4f4f5a412393c24 Test the patch Access Site administration > Courses > Restore course. Upload the backup from the attachment ( backup-moodle2-course-5-mock_course-20210204-0456-nf.mbz ). Go through the restore pages and perform restore. Confirm , the course is restored. Access the course and open Some file resource. Confirm , the file content is displayed correctly.
    • Affected Branches:
      MOODLE_310_STABLE, MOODLE_311_STABLE, MOODLE_39_STABLE, MOODLE_400_STABLE
    • Fixed Branches:
      MOODLE_310_STABLE, MOODLE_39_STABLE
    • Pull from Repository:
    • Pull 3.9 Branch:
      MOODLE_39_MDL-70752
    • Pull 3.10 Branch:
      MOODLE_310_MDL-70752
    • Pull 3.11 Branch:
      MOODLE_311_MDL-70752
    • Pull Master Branch:
      master-MDL-70752

      Description

      The edge case:

      • You have a backup which was created with "no files" option AND
      • The backup is referencing to some file AND
      • You do not have this file in your local filedir AND
      • You do not have any records in mdl_files table for this file AND
      • You have a remote storage configured which has the file AND
      • You are trying to restore the backup.

      As a result, restore fails with the "Unknown exception related to local files (Can not fetch file form URL)" error because of this call https://github.com/moodle/moodle/blob/9dabd071fe7c09ba10f0a7a7b7c7d8164bca9c95/backup/util/dbops/restore_dbops.class.php#L1082

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              mikhailgolenkov Mikhail Golenkov
              Reporter:
              tomotsuyuki Tomo Tsuyuki
              Peer reviewer:
              Brendan Heywood Brendan Heywood
              Integrator:
              Víctor Déniz Falcón Víctor Déniz Falcón
              Tester:
              Ferran Recio Ferran Recio
              Participants:
              Component watchers:
              Matteo Scaramuccia, Andrew Lyons, Dongsheng Cai, Huong Nguyen, Jun Pataleta, Michael Hawkins, Shamim Rezaie, Simey Lameze
              Votes:
              1 Vote for this issue
              Watchers:
              11 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:
                Fix Release Date:
                8/Mar/21

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 6 hours
                  6h