Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-30962

possible improvement to fail proof backup & restore of very large files (2GB+)

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 2.2
    • Fix Version/s: DEV backlog
    • Component/s: Backup
    • Labels:
    • Affected Branches:
      MOODLE_22_STABLE

      Description

      As uploading podcasts, screencasts, lecture recordings, etc., becomes a common practice, a course site can grow to a rather large size, approaching or over the PHP, zip library, and OS file/memory system limit (2GB currently for many cases). This causes backup to fail with no workaround, espeically in the new 2.x file system where it's not easy or practical to manually manage large/many files on the server.

      One way to work around the size limit issue during backuop is to split the resulting zip file into multiple volumes (.m01, .m02...) during zip and read from them with verification at restore. Admins can set a size threshold setting in admin\backup for volume splitting to kick in, such as 1.9 GB or a convenient size for download.

      Some issues with using multi-volume backup files:
      1. backup file portability
      Since the PHP zip library does not support multi-volume, a quick way to achieve zip splitting is to use a multi-volume capable command line zip tool to zip/unzip, such as 7zip/p7zip. However this is a 3rd-party dependency and cannot guarantee cross-instance or cross-LMS compatibility. Implementing volume splitting manually in Moodle is an option, but probably less elegant.

      2. backup system change
      Volume-splitting means multiple files with sequencial numbering, such as .mbs(Moodle Backup Set?), .m01, .m02, and so on. Some modifications to the backup related database schema, backup/restore code, and UI change are needed to understand and work with multi-volume zip sets.

      Sooner(the 2.x file system) or later(force users to upload smaller/less files) this issue will be more widespread and affecting more users. Some change needs to happen in Moodle to save people from coming up with "creative" ways of managing their sites and servers.

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                6 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated: