Moodle
  1. Moodle
  2. MDL-28346

Performing a backup fails when a file is missing

    Details

    • Testing Instructions:
      Hide

      Test pre-requisites

      • A course
      • All file used must be different
      • A forum activity with several posts
        • One post with an image in description (call the file forum_desc.jpg)
        • One post with a file as attachment (call the file forum_attach.txt)
      • A file resource activity with a file in it (call the file file_attach.txt)
      • You can receive emails sent to the admin
      • The tests have to be done one after the other

      Test #1

      1. Navigate to Home ► Courses ► YOUR CATEGORY ► YOUR COURSE ► Backup
      2. Create a backup of the course (leave settings as default)
      3. Make sure the backup finishes without warnings
      4. Save the new backup file for later use, I will call it test1.mbz

      Test #2

      1. Delete the files from moodledata (see procedure below)
      2. Backup your course
      3. Make sure the backup does not fail
      4. Make sure a warning message is printed out to tell you that some files were missing
      5. Save the new backup file for later use, I will call it test2.mbz

      Test #3

      1. Run the `php admin/cli/automated_backups.php`
      2. Make sure the backup ends without unexpected errors
      3. Make sure the console output display information about the missing files
      4. Navigate to Home ► Site administration ► Reports ► Backups
      5. Make sure the course backup status is 'Warning'
      6. Make sure the email sent to the admin displays that a backup had warnings

      Test #4

      1. Navigate to Home ► Courses ► YOUR CATEGORY ► YOUR COURSE ► Restore
      2. Use test2.mbz to restore the course into a new course
      3. Make sure the restore succeeded
      4. Make sure a warning message told you that some files were missing

      Test #5

      1. Navigate to Home ► Courses ► YOUR CATEGORY ► YOUR COURSE ► Restore
      2. Use test1.mbz to restore the course into a new course
      3. Make sure the restore succeeded without any warning

      Delete the files from moodledata

      1. Browse your table mdl_files ordered by ID DESC
      2. Look for the rows matching the file names as specified the pre-requisites (forum_attach, forum_desc, file_attach)
      3. Note down the contenthash for each of them, ie.: da39a3ee5e6b4b0d3255bfef95601890afd80709
      4. Browse to the directories $CFG->dataroot/filedir/da/39 (using the 4 first letters of the contenthash)
      5. Rename the file named as the contenthash. ie.: da39a3ee5e6b4b0d3255bfef95601890afd80709.DELETED
      6. You're done!
      Show
      Test pre-requisites A course All file used must be different A forum activity with several posts One post with an image in description (call the file forum_desc .jpg) One post with a file as attachment (call the file forum_attach .txt) A file resource activity with a file in it (call the file file_attach .txt) You can receive emails sent to the admin The tests have to be done one after the other Test #1 Navigate to Home ► Courses ► YOUR CATEGORY ► YOUR COURSE ► Backup Create a backup of the course (leave settings as default) Make sure the backup finishes without warnings Save the new backup file for later use, I will call it test1.mbz Test #2 Delete the files from moodledata (see procedure below) Backup your course Make sure the backup does not fail Make sure a warning message is printed out to tell you that some files were missing Save the new backup file for later use, I will call it test2.mbz Test #3 Run the `php admin/cli/automated_backups.php` Make sure the backup ends without unexpected errors Make sure the console output display information about the missing files Navigate to Home ► Site administration ► Reports ► Backups Make sure the course backup status is 'Warning' Make sure the email sent to the admin displays that a backup had warnings Test #4 Navigate to Home ► Courses ► YOUR CATEGORY ► YOUR COURSE ► Restore Use test2.mbz to restore the course into a new course Make sure the restore succeeded Make sure a warning message told you that some files were missing Test #5 Navigate to Home ► Courses ► YOUR CATEGORY ► YOUR COURSE ► Restore Use test1.mbz to restore the course into a new course Make sure the restore succeeded without any warning Delete the files from moodledata Browse your table mdl_files ordered by ID DESC Look for the rows matching the file names as specified the pre-requisites ( forum_attach , forum_desc , file_attach ) Note down the contenthash for each of them, ie.: da39a3ee5e6b4b0d3255bfef95601890afd80709 Browse to the directories $CFG->dataroot/filedir/da/39 (using the 4 first letters of the contenthash) Rename the file named as the contenthash. ie.: da39a3ee5e6b4b0d3255bfef95601890afd80709.DELETED You're done!
    • Affected Branches:
      MOODLE_20_STABLE, MOODLE_21_STABLE, MOODLE_22_STABLE, MOODLE_23_STABLE, MOODLE_24_STABLE
    • Fixed Branches:
      MOODLE_22_STABLE, MOODLE_23_STABLE
    • Pull from Repository:
    • Pull Master Branch:
      MDL-28346-master
    • Rank:
      17990

      Description

      My language is Portuguese of Brazil, or will post in both languages.
      I have problems to build up some courses.
      I tried every way possible before resorting to you, and I saw that I can no longer solve this problem.
      I have the moodle distribution of fully intact, without code modifications, or were talking to a native installation files.
      Summarizing the problem, for some courses I run the backup normally, but for others I see that the directory "/ moodledata / temp / backup" files it generates, but it seems he can not direct the file "*. nu.mbz" to the database by stopping the process.
      I made the migration from version 1.9.7 to 2.0.2
      I use postgres 8.3, Moodle 2.0.2, php 5.3.8
      Below lay the screen printing, which are the errors in detail with the debugger enabled function

      Please ask for special attention, because I saw some post in the forums related solutions fancy, and using Moodle for teaching instuição over 3000 subjects and we have created this solution as soon as possible.

      Regards

      --------------------------------------------------

      Meu idioma é português do Brasil, ou seja irei postar nas duas línguas.
      Estou com problemas para criar backup de alguns cursos.
      Tentei todas as formas possíveis antes de recorrer para vocês, e acabei vendo que não consigo mais resolver este problema.
      Tenho a distribuição do moodle totalmente intacta, sem modificações no código, ou seja estavamos falando de uma instalação com arquivos nativos.
      Resumindo o problema, para alguns cursos consigo gerar o backup normalmente, porém para outros vejo que no diretório "/moodledata/temp/backup" ele gera os arquivos, porém parece que ele não consegue direcionar o arquivo "*.nu.mbz" para o banco de dados, parando o processo.
      Fiz a migração da versão 1.9.7 para 2.0.2
      Utilizo postgres 8.3, moodle 2.0.2, php 5.3.8
      Abaixo coloco as impressão da tela, na qual estão os erros detalhadamente com a função de debbug ativada

      Por favor peço uma atenção especial, pois vi alguns post relacionados nos fóruns com soluções mirabolantes, sendo que utilizamos o moodle para instuição de ensino com mais de 3000 disciplinas criadas e precisamos desta solução o quanto antes.

      Abraço,

        Issue Links

          Activity

          Sergio created issue -
          Andreas Grabs made changes -
          Field Original Value New Value
          Testing Instructions Stack trace:

              line 174 of /lib/filestorage/stored_file.php: file_exception thrown
              line 90 of /backup/util/helper/backup_file_manager.class.php: call to stored_file->copy_content_to()
              line 101 of /backup/moodle2/backup_custom_fields.php: call to backup_file_manager::copy_file_moodle2backup()
              line 70 of /backup/util/structure/backup_nested_element.class.php: call to file_nested_element->fill_values()
              line 94 of /backup/moodle2/backup_custom_fields.php: call to backup_nested_element->process()
              line 95 of /backup/util/structure/backup_nested_element.class.php: call to file_nested_element->process()
              line 95 of /backup/util/plan/backup_structure_step.class.php: call to backup_nested_element->process()
              line 153 of /backup/util/plan/base_task.class.php: call to backup_structure_step->execute()
              line 148 of /backup/util/plan/base_plan.class.php: call to base_task->execute()
              line 105 of /backup/util/plan/backup_plan.class.php: call to base_plan->execute()
              line 293 of /backup/controller/backup_controller.class.php: call to backup_plan->execute()
              line 111 of /backup/util/ui/backup_ui.class.php: call to backup_controller->execute_plan()
              line 87 of /backup/backup.php: call to backup_ui->execute(
          # create a course
          # create a resource or any activity with files support
          # attach a file to this resource
          # look at the database in the table mdl_files for that file and memorize the hashvalue of that file
          # go to the moodledata/filedir and delete or rename that file so moodle think it is vanished
          # go back to the course and perform a backup
          the following error message will be prompted.
          ------------------------------------------------
          Can not read file, either file does not exist or there are permission problems

          More information about this error
          Debug info: /home/www/grabs-edv.com/files/moodle/filedir/54/8a/548a1fb742c6b54c0f194336f350ef1574079448
          Stack trace:

              line 174 of /lib/filestorage/stored_file.php: file_exception thrown
              line 90 of /backup/util/helper/backup_file_manager.class.php: call to stored_file->copy_content_to()
              line 101 of /backup/moodle2/backup_custom_fields.php: call to backup_file_manager::copy_file_moodle2backup()
              line 70 of /backup/util/structure/backup_nested_element.class.php: call to file_nested_element->fill_values()
              line 94 of /backup/moodle2/backup_custom_fields.php: call to backup_nested_element->process()
              line 95 of /backup/util/structure/backup_nested_element.class.php: call to file_nested_element->process()
              line 95 of /backup/util/plan/backup_structure_step.class.php: call to backup_nested_element->process()
              line 153 of /backup/util/plan/base_task.class.php: call to backup_structure_step->execute()
              line 148 of /backup/util/plan/base_plan.class.php: call to base_task->execute()
              line 105 of /backup/util/plan/backup_plan.class.php: call to base_plan->execute()
              line 296 of /backup/controller/backup_controller.class.php: call to backup_plan->execute()
              line 111 of /backup/util/ui/backup_ui.class.php: call to backup_controller->execute_plan()
              line 89 of /backup/backup.php: call to backup_ui->execute()
          Priority Minor [ 4 ] Critical [ 2 ]
          Andreas Grabs made changes -
          Affects Version/s 2.1.3 [ 11251 ]
          Affects Version/s 2.2 [ 10656 ]
          Helen Foster made changes -
          Fix Version/s STABLE backlog [ 10463 ]
          Labels triaged
          Brian King made changes -
          Labels triaged partner triaged
          Frédéric Massart made changes -
          Assignee Eloy Lafuente (stronk7) [ stronk7 ] Frédéric Massart [ fred ]
          Frédéric Massart made changes -
          Fix Version/s STABLE Sprint 21 [ 12155 ]
          Fix Version/s STABLE backlog [ 10463 ]
          Frédéric Massart made changes -
          Status Open [ 1 ] Development in progress [ 3 ]
          Frédéric Massart made changes -
          Testing Instructions # create a course
          # create a resource or any activity with files support
          # attach a file to this resource
          # look at the database in the table mdl_files for that file and memorize the hashvalue of that file
          # go to the moodledata/filedir and delete or rename that file so moodle think it is vanished
          # go back to the course and perform a backup
          the following error message will be prompted.
          ------------------------------------------------
          Can not read file, either file does not exist or there are permission problems

          More information about this error
          Debug info: /home/www/grabs-edv.com/files/moodle/filedir/54/8a/548a1fb742c6b54c0f194336f350ef1574079448
          Stack trace:

              line 174 of /lib/filestorage/stored_file.php: file_exception thrown
              line 90 of /backup/util/helper/backup_file_manager.class.php: call to stored_file->copy_content_to()
              line 101 of /backup/moodle2/backup_custom_fields.php: call to backup_file_manager::copy_file_moodle2backup()
              line 70 of /backup/util/structure/backup_nested_element.class.php: call to file_nested_element->fill_values()
              line 94 of /backup/moodle2/backup_custom_fields.php: call to backup_nested_element->process()
              line 95 of /backup/util/structure/backup_nested_element.class.php: call to file_nested_element->process()
              line 95 of /backup/util/plan/backup_structure_step.class.php: call to backup_nested_element->process()
              line 153 of /backup/util/plan/base_task.class.php: call to backup_structure_step->execute()
              line 148 of /backup/util/plan/base_plan.class.php: call to base_task->execute()
              line 105 of /backup/util/plan/backup_plan.class.php: call to base_plan->execute()
              line 296 of /backup/controller/backup_controller.class.php: call to backup_plan->execute()
              line 111 of /backup/util/ui/backup_ui.class.php: call to backup_controller->execute_plan()
              line 89 of /backup/backup.php: call to backup_ui->execute()
          # Create a course
          # Create a resource or any activity with files support
          # Attach a file to this resource
          # Look at the database in the table mdl_files for that file and memorize the hashvalue of that file
          # Go to the moodledata/filedir and delete or rename that file so moodle think it is vanished
          # Go back to the course and perform a backup, make sure the backup does not fail
          # Go to restore the course in a new one, make sure the restore works
          Frédéric Massart made changes -
          Summary Can not read the file, the file does not exist or no permission issues Performing a backup fails when a file is missing
          Frédéric Massart made changes -
          Frédéric Massart made changes -
          Affects Version/s 2.3 [ 10657 ]
          Michael de Raadt made changes -
          Fix Version/s STABLE Sprint 22 [ 12156 ]
          Fix Version/s STABLE Sprint 21 [ 12155 ]
          Frédéric Massart made changes -
          Status Development in progress [ 3 ] Waiting for peer review [ 10012 ]
          David Monllaó made changes -
          Original Estimate 0 minutes [ 0 ]
          Remaining Estimate 0 minutes [ 0 ]
          Status Waiting for peer review [ 10012 ] Peer review in progress [ 10013 ]
          Peer reviewer davmon
          David Monllaó made changes -
          Peer reviewer davmon
          Frédéric Massart made changes -
          Status Peer review in progress [ 10013 ] Development in progress [ 3 ]
          Frédéric Massart made changes -
          Status Development in progress [ 3 ] Waiting for peer review [ 10012 ]
          Martin Dougiamas made changes -
          Priority Critical [ 2 ] Blocker [ 1 ]
          Michael de Raadt made changes -
          Fix Version/s STABLE Sprint 23 Alpha [ 12358 ]
          Fix Version/s STABLE Sprint 22 [ 12156 ]
          Frédéric Massart made changes -
          Fix Version/s STABLE Sprint 23 Omega [ 12362 ]
          Fix Version/s STABLE Sprint 23 Alpha [ 12358 ]
          Eloy Lafuente (stronk7) made changes -
          Status Waiting for peer review [ 10012 ] Peer review in progress [ 10013 ]
          Peer reviewer stronk7
          Frédéric Massart made changes -
          Testing Instructions # Create a course
          # Create a resource or any activity with files support
          # Attach a file to this resource
          # Look at the database in the table mdl_files for that file and memorize the hashvalue of that file
          # Go to the moodledata/filedir and delete or rename that file so moodle think it is vanished
          # Go back to the course and perform a backup, make sure the backup does not fail
          # Go to restore the course in a new one, make sure the restore works
          *Test pre-requisites*

          - A course
          - All file used must be different
          - A forum activity with several posts
          -- One post with an image in description (call the file _forum_desc_.jpg)
          -- One post with a file as attachment (call the file _forum_attach_.txt)
          - A file resource activity with a file in it (call the file _file_attach_.txt)
          - You can receive emails sent to the admin
          - The tests have to be done one after the other

          *Test #1*

          # Navigate to Home ► Courses ► YOUR CATEGORY ► YOUR COURSE ► Backup
          # Create a backup of the course (leave settings as default)
          # *Make sure* the backup finishes without warnings
          # Save the new backup file for later use, I will call it _test1.mbz_

          *Test #2*

          # Delete the files from moodledata (see procedure below)
          # Backup your course
          # *Make sure* the backup does not fail
          # *Make sure* a warning message is printed out to tell you that some files were missing
          # Save the new backup file for later use, I will call it _test2.mbz_

          *Test #3*

          # Run the `php admin/cli/automated_backups.php`
          # *Make sure* the backup ends without unexpected errors
          # *Make sure* the console output display information about the missing files
          # Navigate to Home ► Site administration ► Reports ► Backups
          # *Make sure* the course backup status is 'Warning'
          # *Make sure* the email sent to the admin displays that a backup had warnings

          *Test #4*

          # Navigate to Home ► Courses ► YOUR CATEGORY ► YOUR COURSE ► Restore
          # Use _test2.mbz_ to restore the course into a new course
          # *Make sure* the restore succeeded
          # *Make sure* a warning message told you that some files were missing

          *Test #5*

          # Navigate to Home ► Courses ► YOUR CATEGORY ► YOUR COURSE ► Restore
          # Use _test1.mbz_ to restore the course into a new course
          # *Make sure* the restore succeeded without any warning

          *Delete the files from moodledata*

          # Browse your table mdl_files ordered by ID DESC
          # Look for the rows matching the file names as specified the pre-requisites (_forum_attach_, _forum_desc_, _file_attach_)
          # Note down the _contenthash_ for each of them, ie.: _da39a3ee5e6b4b0d3255bfef95601890afd80709_
          # Browse to the directories $CFG->dataroot/filedir/da/39 (using the 4 first letters of the contenthash)
          # Rename the file named as the contenthash. ie.: _da39a3ee5e6b4b0d3255bfef95601890afd80709.DELETED_
          # You're done!
          Michael de Raadt made changes -
          Link This issue has been marked as being related by MDL-35019 [ MDL-35019 ]
          Eloy Lafuente (stronk7) made changes -
          Status Peer review in progress [ 10013 ] Development in progress [ 3 ]
          Frédéric Massart made changes -
          Status Development in progress [ 3 ] Waiting for peer review [ 10012 ]
          Frédéric Massart made changes -
          Link This issue has a non-specific relationship to MDL-35075 [ MDL-35075 ]
          Eloy Lafuente (stronk7) made changes -
          Status Waiting for peer review [ 10012 ] Waiting for integration review [ 10010 ]
          Sam Hemelryk made changes -
          Currently in integration Yes [ 10041 ]
          Eloy Lafuente (stronk7) made changes -
          Labels partner triaged partner triaged ui_change
          Eloy Lafuente (stronk7) made changes -
          Labels partner triaged ui_change docs_required partner triaged ui_change
          Eloy Lafuente (stronk7) made changes -
          Status Waiting for integration review [ 10010 ] Integration review in progress [ 10004 ]
          Integrator stronk7
          Eloy Lafuente (stronk7) made changes -
          Status Integration review in progress [ 10004 ] Waiting for testing [ 10005 ]
          Affects Version/s 2.4 [ 12255 ]
          Fix Version/s 2.2.5 [ 12352 ]
          Fix Version/s 2.3.2 [ 12353 ]
          Michael de Raadt made changes -
          Tester abgreeve
          Adrian Greeve made changes -
          Status Waiting for testing [ 10005 ] Testing in progress [ 10011 ]
          Adrian Greeve made changes -
          Status Testing in progress [ 10011 ] Tested [ 10006 ]
          Eloy Lafuente (stronk7) made changes -
          Status Tested [ 10006 ] Closed [ 6 ]
          Resolution Fixed [ 1 ]
          Currently in integration Yes [ 10041 ]
          Integration date 07/Sep/12
          Mary Cooch made changes -
          Labels docs_required partner triaged ui_change partner triaged ui_change
          Eloy Lafuente (stronk7) made changes -
          Fix Version/s STABLE Sprint 23 Omega [ 12362 ]

            People

            • Votes:
              11 Vote for this issue
              Watchers:
              17 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: