Moodle
  1. Moodle
  2. MDL-28346

Performing a backup fails when a file is missing

    Details

    • Testing Instructions:
      Hide

      Test pre-requisites

      • A course
      • All file used must be different
      • A forum activity with several posts
        • One post with an image in description (call the file forum_desc.jpg)
        • One post with a file as attachment (call the file forum_attach.txt)
      • A file resource activity with a file in it (call the file file_attach.txt)
      • You can receive emails sent to the admin
      • The tests have to be done one after the other

      Test #1

      1. Navigate to Home ► Courses ► YOUR CATEGORY ► YOUR COURSE ► Backup
      2. Create a backup of the course (leave settings as default)
      3. Make sure the backup finishes without warnings
      4. Save the new backup file for later use, I will call it test1.mbz

      Test #2

      1. Delete the files from moodledata (see procedure below)
      2. Backup your course
      3. Make sure the backup does not fail
      4. Make sure a warning message is printed out to tell you that some files were missing
      5. Save the new backup file for later use, I will call it test2.mbz

      Test #3

      1. Run the `php admin/cli/automated_backups.php`
      2. Make sure the backup ends without unexpected errors
      3. Make sure the console output display information about the missing files
      4. Navigate to Home ► Site administration ► Reports ► Backups
      5. Make sure the course backup status is 'Warning'
      6. Make sure the email sent to the admin displays that a backup had warnings

      Test #4

      1. Navigate to Home ► Courses ► YOUR CATEGORY ► YOUR COURSE ► Restore
      2. Use test2.mbz to restore the course into a new course
      3. Make sure the restore succeeded
      4. Make sure a warning message told you that some files were missing

      Test #5

      1. Navigate to Home ► Courses ► YOUR CATEGORY ► YOUR COURSE ► Restore
      2. Use test1.mbz to restore the course into a new course
      3. Make sure the restore succeeded without any warning

      Delete the files from moodledata

      1. Browse your table mdl_files ordered by ID DESC
      2. Look for the rows matching the file names as specified the pre-requisites (forum_attach, forum_desc, file_attach)
      3. Note down the contenthash for each of them, ie.: da39a3ee5e6b4b0d3255bfef95601890afd80709
      4. Browse to the directories $CFG->dataroot/filedir/da/39 (using the 4 first letters of the contenthash)
      5. Rename the file named as the contenthash. ie.: da39a3ee5e6b4b0d3255bfef95601890afd80709.DELETED
      6. You're done!
      Show
      Test pre-requisites A course All file used must be different A forum activity with several posts One post with an image in description (call the file forum_desc .jpg) One post with a file as attachment (call the file forum_attach .txt) A file resource activity with a file in it (call the file file_attach .txt) You can receive emails sent to the admin The tests have to be done one after the other Test #1 Navigate to Home ► Courses ► YOUR CATEGORY ► YOUR COURSE ► Backup Create a backup of the course (leave settings as default) Make sure the backup finishes without warnings Save the new backup file for later use, I will call it test1.mbz Test #2 Delete the files from moodledata (see procedure below) Backup your course Make sure the backup does not fail Make sure a warning message is printed out to tell you that some files were missing Save the new backup file for later use, I will call it test2.mbz Test #3 Run the `php admin/cli/automated_backups.php` Make sure the backup ends without unexpected errors Make sure the console output display information about the missing files Navigate to Home ► Site administration ► Reports ► Backups Make sure the course backup status is 'Warning' Make sure the email sent to the admin displays that a backup had warnings Test #4 Navigate to Home ► Courses ► YOUR CATEGORY ► YOUR COURSE ► Restore Use test2.mbz to restore the course into a new course Make sure the restore succeeded Make sure a warning message told you that some files were missing Test #5 Navigate to Home ► Courses ► YOUR CATEGORY ► YOUR COURSE ► Restore Use test1.mbz to restore the course into a new course Make sure the restore succeeded without any warning Delete the files from moodledata Browse your table mdl_files ordered by ID DESC Look for the rows matching the file names as specified the pre-requisites ( forum_attach , forum_desc , file_attach ) Note down the contenthash for each of them, ie.: da39a3ee5e6b4b0d3255bfef95601890afd80709 Browse to the directories $CFG->dataroot/filedir/da/39 (using the 4 first letters of the contenthash) Rename the file named as the contenthash. ie.: da39a3ee5e6b4b0d3255bfef95601890afd80709.DELETED You're done!
    • Affected Branches:
      MOODLE_20_STABLE, MOODLE_21_STABLE, MOODLE_22_STABLE, MOODLE_23_STABLE, MOODLE_24_STABLE
    • Fixed Branches:
      MOODLE_22_STABLE, MOODLE_23_STABLE
    • Pull from Repository:
    • Pull Master Branch:
      MDL-28346-master

      Description

      My language is Portuguese of Brazil, or will post in both languages.
      I have problems to build up some courses.
      I tried every way possible before resorting to you, and I saw that I can no longer solve this problem.
      I have the moodle distribution of fully intact, without code modifications, or were talking to a native installation files.
      Summarizing the problem, for some courses I run the backup normally, but for others I see that the directory "/ moodledata / temp / backup" files it generates, but it seems he can not direct the file "*. nu.mbz" to the database by stopping the process.
      I made the migration from version 1.9.7 to 2.0.2
      I use postgres 8.3, Moodle 2.0.2, php 5.3.8
      Below lay the screen printing, which are the errors in detail with the debugger enabled function

      Please ask for special attention, because I saw some post in the forums related solutions fancy, and using Moodle for teaching instuição over 3000 subjects and we have created this solution as soon as possible.

      Regards

      --------------------------------------------------

      Meu idioma é português do Brasil, ou seja irei postar nas duas línguas.
      Estou com problemas para criar backup de alguns cursos.
      Tentei todas as formas possíveis antes de recorrer para vocês, e acabei vendo que não consigo mais resolver este problema.
      Tenho a distribuição do moodle totalmente intacta, sem modificações no código, ou seja estavamos falando de uma instalação com arquivos nativos.
      Resumindo o problema, para alguns cursos consigo gerar o backup normalmente, porém para outros vejo que no diretório "/moodledata/temp/backup" ele gera os arquivos, porém parece que ele não consegue direcionar o arquivo "*.nu.mbz" para o banco de dados, parando o processo.
      Fiz a migração da versão 1.9.7 para 2.0.2
      Utilizo postgres 8.3, moodle 2.0.2, php 5.3.8
      Abaixo coloco as impressão da tela, na qual estão os erros detalhadamente com a função de debbug ativada

      Por favor peço uma atenção especial, pois vi alguns post relacionados nos fóruns com soluções mirabolantes, sendo que utilizamos o moodle para instuição de ensino com mais de 3000 disciplinas criadas e precisamos desta solução o quanto antes.

      Abraço,

        Gliffy Diagrams

          Issue Links

            Activity

            Sergio created issue -
            Andreas Grabs made changes -
            Field Original Value New Value
            Testing Instructions Stack trace:

                line 174 of /lib/filestorage/stored_file.php: file_exception thrown
                line 90 of /backup/util/helper/backup_file_manager.class.php: call to stored_file->copy_content_to()
                line 101 of /backup/moodle2/backup_custom_fields.php: call to backup_file_manager::copy_file_moodle2backup()
                line 70 of /backup/util/structure/backup_nested_element.class.php: call to file_nested_element->fill_values()
                line 94 of /backup/moodle2/backup_custom_fields.php: call to backup_nested_element->process()
                line 95 of /backup/util/structure/backup_nested_element.class.php: call to file_nested_element->process()
                line 95 of /backup/util/plan/backup_structure_step.class.php: call to backup_nested_element->process()
                line 153 of /backup/util/plan/base_task.class.php: call to backup_structure_step->execute()
                line 148 of /backup/util/plan/base_plan.class.php: call to base_task->execute()
                line 105 of /backup/util/plan/backup_plan.class.php: call to base_plan->execute()
                line 293 of /backup/controller/backup_controller.class.php: call to backup_plan->execute()
                line 111 of /backup/util/ui/backup_ui.class.php: call to backup_controller->execute_plan()
                line 87 of /backup/backup.php: call to backup_ui->execute(
            # create a course
            # create a resource or any activity with files support
            # attach a file to this resource
            # look at the database in the table mdl_files for that file and memorize the hashvalue of that file
            # go to the moodledata/filedir and delete or rename that file so moodle think it is vanished
            # go back to the course and perform a backup
            the following error message will be prompted.
            ------------------------------------------------
            Can not read file, either file does not exist or there are permission problems

            More information about this error
            Debug info: /home/www/grabs-edv.com/files/moodle/filedir/54/8a/548a1fb742c6b54c0f194336f350ef1574079448
            Stack trace:

                line 174 of /lib/filestorage/stored_file.php: file_exception thrown
                line 90 of /backup/util/helper/backup_file_manager.class.php: call to stored_file->copy_content_to()
                line 101 of /backup/moodle2/backup_custom_fields.php: call to backup_file_manager::copy_file_moodle2backup()
                line 70 of /backup/util/structure/backup_nested_element.class.php: call to file_nested_element->fill_values()
                line 94 of /backup/moodle2/backup_custom_fields.php: call to backup_nested_element->process()
                line 95 of /backup/util/structure/backup_nested_element.class.php: call to file_nested_element->process()
                line 95 of /backup/util/plan/backup_structure_step.class.php: call to backup_nested_element->process()
                line 153 of /backup/util/plan/base_task.class.php: call to backup_structure_step->execute()
                line 148 of /backup/util/plan/base_plan.class.php: call to base_task->execute()
                line 105 of /backup/util/plan/backup_plan.class.php: call to base_plan->execute()
                line 296 of /backup/controller/backup_controller.class.php: call to backup_plan->execute()
                line 111 of /backup/util/ui/backup_ui.class.php: call to backup_controller->execute_plan()
                line 89 of /backup/backup.php: call to backup_ui->execute()
            Priority Minor [ 4 ] Critical [ 2 ]
            Andreas Grabs made changes -
            Affects Version/s 2.1.3 [ 11251 ]
            Affects Version/s 2.2 [ 10656 ]
            Helen Foster made changes -
            Fix Version/s STABLE backlog [ 10463 ]
            Labels triaged
            Brian King made changes -
            Labels triaged partner triaged
            Frédéric Massart made changes -
            Assignee Eloy Lafuente (stronk7) [ stronk7 ] Frédéric Massart [ fred ]
            Frédéric Massart made changes -
            Fix Version/s STABLE Sprint 21 [ 12155 ]
            Fix Version/s STABLE backlog [ 10463 ]
            Frédéric Massart made changes -
            Status Open [ 1 ] Development in progress [ 3 ]
            Frédéric Massart made changes -
            Testing Instructions # create a course
            # create a resource or any activity with files support
            # attach a file to this resource
            # look at the database in the table mdl_files for that file and memorize the hashvalue of that file
            # go to the moodledata/filedir and delete or rename that file so moodle think it is vanished
            # go back to the course and perform a backup
            the following error message will be prompted.
            ------------------------------------------------
            Can not read file, either file does not exist or there are permission problems

            More information about this error
            Debug info: /home/www/grabs-edv.com/files/moodle/filedir/54/8a/548a1fb742c6b54c0f194336f350ef1574079448
            Stack trace:

                line 174 of /lib/filestorage/stored_file.php: file_exception thrown
                line 90 of /backup/util/helper/backup_file_manager.class.php: call to stored_file->copy_content_to()
                line 101 of /backup/moodle2/backup_custom_fields.php: call to backup_file_manager::copy_file_moodle2backup()
                line 70 of /backup/util/structure/backup_nested_element.class.php: call to file_nested_element->fill_values()
                line 94 of /backup/moodle2/backup_custom_fields.php: call to backup_nested_element->process()
                line 95 of /backup/util/structure/backup_nested_element.class.php: call to file_nested_element->process()
                line 95 of /backup/util/plan/backup_structure_step.class.php: call to backup_nested_element->process()
                line 153 of /backup/util/plan/base_task.class.php: call to backup_structure_step->execute()
                line 148 of /backup/util/plan/base_plan.class.php: call to base_task->execute()
                line 105 of /backup/util/plan/backup_plan.class.php: call to base_plan->execute()
                line 296 of /backup/controller/backup_controller.class.php: call to backup_plan->execute()
                line 111 of /backup/util/ui/backup_ui.class.php: call to backup_controller->execute_plan()
                line 89 of /backup/backup.php: call to backup_ui->execute()
            # Create a course
            # Create a resource or any activity with files support
            # Attach a file to this resource
            # Look at the database in the table mdl_files for that file and memorize the hashvalue of that file
            # Go to the moodledata/filedir and delete or rename that file so moodle think it is vanished
            # Go back to the course and perform a backup, make sure the backup does not fail
            # Go to restore the course in a new one, make sure the restore works
            Frédéric Massart made changes -
            Summary Can not read the file, the file does not exist or no permission issues Performing a backup fails when a file is missing
            Frédéric Massart made changes -
            Frédéric Massart made changes -
            Affects Version/s 2.3 [ 10657 ]
            Michael de Raadt made changes -
            Fix Version/s STABLE Sprint 22 [ 12156 ]
            Fix Version/s STABLE Sprint 21 [ 12155 ]
            Frédéric Massart made changes -
            Status Development in progress [ 3 ] Waiting for peer review [ 10012 ]
            David Monllaó made changes -
            Original Estimate 0 minutes [ 0 ]
            Remaining Estimate 0 minutes [ 0 ]
            Status Waiting for peer review [ 10012 ] Peer review in progress [ 10013 ]
            Peer reviewer davmon
            David Monllaó made changes -
            Peer reviewer davmon
            Frédéric Massart made changes -
            Status Peer review in progress [ 10013 ] Development in progress [ 3 ]
            Frédéric Massart made changes -
            Status Development in progress [ 3 ] Waiting for peer review [ 10012 ]
            Martin Dougiamas made changes -
            Priority Critical [ 2 ] Blocker [ 1 ]
            Michael de Raadt made changes -
            Fix Version/s STABLE Sprint 23 Alpha [ 12358 ]
            Fix Version/s STABLE Sprint 22 [ 12156 ]
            Frédéric Massart made changes -
            Fix Version/s STABLE Sprint 23 Omega [ 12362 ]
            Fix Version/s STABLE Sprint 23 Alpha [ 12358 ]
            Eloy Lafuente (stronk7) made changes -
            Status Waiting for peer review [ 10012 ] Peer review in progress [ 10013 ]
            Peer reviewer stronk7
            Frédéric Massart made changes -
            Testing Instructions # Create a course
            # Create a resource or any activity with files support
            # Attach a file to this resource
            # Look at the database in the table mdl_files for that file and memorize the hashvalue of that file
            # Go to the moodledata/filedir and delete or rename that file so moodle think it is vanished
            # Go back to the course and perform a backup, make sure the backup does not fail
            # Go to restore the course in a new one, make sure the restore works
            *Test pre-requisites*

            - A course
            - All file used must be different
            - A forum activity with several posts
            -- One post with an image in description (call the file _forum_desc_.jpg)
            -- One post with a file as attachment (call the file _forum_attach_.txt)
            - A file resource activity with a file in it (call the file _file_attach_.txt)
            - You can receive emails sent to the admin
            - The tests have to be done one after the other

            *Test #1*

            # Navigate to Home ► Courses ► YOUR CATEGORY ► YOUR COURSE ► Backup
            # Create a backup of the course (leave settings as default)
            # *Make sure* the backup finishes without warnings
            # Save the new backup file for later use, I will call it _test1.mbz_

            *Test #2*

            # Delete the files from moodledata (see procedure below)
            # Backup your course
            # *Make sure* the backup does not fail
            # *Make sure* a warning message is printed out to tell you that some files were missing
            # Save the new backup file for later use, I will call it _test2.mbz_

            *Test #3*

            # Run the `php admin/cli/automated_backups.php`
            # *Make sure* the backup ends without unexpected errors
            # *Make sure* the console output display information about the missing files
            # Navigate to Home ► Site administration ► Reports ► Backups
            # *Make sure* the course backup status is 'Warning'
            # *Make sure* the email sent to the admin displays that a backup had warnings

            *Test #4*

            # Navigate to Home ► Courses ► YOUR CATEGORY ► YOUR COURSE ► Restore
            # Use _test2.mbz_ to restore the course into a new course
            # *Make sure* the restore succeeded
            # *Make sure* a warning message told you that some files were missing

            *Test #5*

            # Navigate to Home ► Courses ► YOUR CATEGORY ► YOUR COURSE ► Restore
            # Use _test1.mbz_ to restore the course into a new course
            # *Make sure* the restore succeeded without any warning

            *Delete the files from moodledata*

            # Browse your table mdl_files ordered by ID DESC
            # Look for the rows matching the file names as specified the pre-requisites (_forum_attach_, _forum_desc_, _file_attach_)
            # Note down the _contenthash_ for each of them, ie.: _da39a3ee5e6b4b0d3255bfef95601890afd80709_
            # Browse to the directories $CFG->dataroot/filedir/da/39 (using the 4 first letters of the contenthash)
            # Rename the file named as the contenthash. ie.: _da39a3ee5e6b4b0d3255bfef95601890afd80709.DELETED_
            # You're done!
            Michael de Raadt made changes -
            Link This issue has been marked as being related by MDL-35019 [ MDL-35019 ]
            Eloy Lafuente (stronk7) made changes -
            Status Peer review in progress [ 10013 ] Development in progress [ 3 ]
            Frédéric Massart made changes -
            Status Development in progress [ 3 ] Waiting for peer review [ 10012 ]
            Frédéric Massart made changes -
            Link This issue has a non-specific relationship to MDL-35075 [ MDL-35075 ]
            Eloy Lafuente (stronk7) made changes -
            Status Waiting for peer review [ 10012 ] Waiting for integration review [ 10010 ]
            Sam Hemelryk made changes -
            Currently in integration Yes [ 10041 ]
            Eloy Lafuente (stronk7) made changes -
            Labels partner triaged partner triaged ui_change
            Eloy Lafuente (stronk7) made changes -
            Labels partner triaged ui_change docs_required partner triaged ui_change
            Eloy Lafuente (stronk7) made changes -
            Status Waiting for integration review [ 10010 ] Integration review in progress [ 10004 ]
            Integrator stronk7
            Eloy Lafuente (stronk7) made changes -
            Status Integration review in progress [ 10004 ] Waiting for testing [ 10005 ]
            Affects Version/s 2.4 [ 12255 ]
            Fix Version/s 2.2.5 [ 12352 ]
            Fix Version/s 2.3.2 [ 12353 ]
            Michael de Raadt made changes -
            Tester abgreeve
            Adrian Greeve made changes -
            Status Waiting for testing [ 10005 ] Testing in progress [ 10011 ]
            Adrian Greeve made changes -
            Status Testing in progress [ 10011 ] Tested [ 10006 ]
            Eloy Lafuente (stronk7) made changes -
            Status Tested [ 10006 ] Closed [ 6 ]
            Resolution Fixed [ 1 ]
            Currently in integration Yes [ 10041 ]
            Integration date 07/Sep/12
            Mary Cooch made changes -
            Labels docs_required partner triaged ui_change partner triaged ui_change
            Eloy Lafuente (stronk7) made changes -
            Fix Version/s STABLE Sprint 23 Omega [ 12362 ]

              People

              • Votes:
                11 Vote for this issue
                Watchers:
                17 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: