Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-30962

possible improvement to fail proof backup & restore of very large files (2GB+)



    • Improvement
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • 2.2
    • DEV backlog
    • Backup


      As uploading podcasts, screencasts, lecture recordings, etc., becomes a common practice, a course site can grow to a rather large size, approaching or over the PHP, zip library, and OS file/memory system limit (2GB currently for many cases). This causes backup to fail with no workaround, espeically in the new 2.x file system where it's not easy or practical to manually manage large/many files on the server.

      One way to work around the size limit issue during backuop is to split the resulting zip file into multiple volumes (.m01, .m02...) during zip and read from them with verification at restore. Admins can set a size threshold setting in admin\backup for volume splitting to kick in, such as 1.9 GB or a convenient size for download.

      Some issues with using multi-volume backup files:
      1. backup file portability
      Since the PHP zip library does not support multi-volume, a quick way to achieve zip splitting is to use a multi-volume capable command line zip tool to zip/unzip, such as 7zip/p7zip. However this is a 3rd-party dependency and cannot guarantee cross-instance or cross-LMS compatibility. Implementing volume splitting manually in Moodle is an option, but probably less elegant.

      2. backup system change
      Volume-splitting means multiple files with sequencial numbering, such as .mbs(Moodle Backup Set?), .m01, .m02, and so on. Some modifications to the backup related database schema, backup/restore code, and UI change are needed to understand and work with multi-volume zip sets.

      Sooner(the 2.x file system) or later(force users to upload smaller/less files) this issue will be more widespread and affecting more users. Some change needs to happen in Moodle to save people from coming up with "creative" ways of managing their sites and servers.


        Issue Links



              moodle.com moodle.com
              davidtang David Tang
              Adrian Greeve, Jake Dallimore, Mathew May, Mihail Geshoski, Sujith Haridasan
              6 Vote for this issue
              6 Start watching this issue