Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-43815

Configurable size limit for "Download all submissions" feature to prevent performance problems

    XMLWordPrintable

    Details

    • Testing Instructions:
      Hide
      • Create one assignment allowing file submissions, and submit a sufficient number of large files so that the aggregate size of all submissions will be sufficiently large (suggest several gigabytes) (or use existing)
      • Create a second assignment with significantly smaller aggregate submission size (or use existing)
      • Ensure assign | downloadall_limit_enabled is checked in admin/settings.php?section=modsettingassign
      • Set assign | downloadall_limit to smaller than the aggregate submission size of the first assignment, larger than size for assignment 2.
      • Click "Download all assignments" for assignment 2 - this should succeed as normal.
      • Click "Download all assignments" for assignment 1 - this should fail, with a suitable message.
      • When clicked, wording for link should change to indicate download has started.
      • Uncheck assign | downloadall_limit_enabled and retry for assignment 1 - this should succeed as normal.
      • Try download of assignment 1 again (with JavaScript enabled). After a short wait, while download is still being processed, click link again. Nothing should happen.
      • Disable JavaScript, repeat above test. You should be forwarded to an error page with a suitable message.
      Show
      Create one assignment allowing file submissions, and submit a sufficient number of large files so that the aggregate size of all submissions will be sufficiently large (suggest several gigabytes) (or use existing) Create a second assignment with significantly smaller aggregate submission size (or use existing) Ensure assign | downloadall_limit_enabled is checked in admin/settings.php?section=modsettingassign Set assign | downloadall_limit to smaller than the aggregate submission size of the first assignment, larger than size for assignment 2. Click "Download all assignments" for assignment 2 - this should succeed as normal. Click "Download all assignments" for assignment 1 - this should fail, with a suitable message. When clicked, wording for link should change to indicate download has started. Uncheck assign | downloadall_limit_enabled and retry for assignment 1 - this should succeed as normal. Try download of assignment 1 again (with JavaScript enabled). After a short wait, while download is still being processed, click link again. Nothing should happen. Disable JavaScript, repeat above test. You should be forwarded to an error page with a suitable message.
    • Affected Branches:
      MOODLE_25_STABLE

      Description

      If an assignment has many large submissions, use of the "Download all submissions" feature in mod_assign can cause high system load, and severely affect system performance (appears to be worse with load-balanced servers using round-robin).

      As creating the zip files can take a long time, the user may become impatient and click the link more than once. This causes multiple partial zip files to be created in dataroot/temp, rapidly filling disk space

      rob@MoodleWeb1pp:/var/www/moodle-data/temp$ ls -lh assignment_*
      rw------ 1 www-data www-data 0 Jan 22 15:34 assignment_7oJUKn
      rw------ 1 www-data www-data 669M Jan 22 15:38 assignment_7oJUKn.OiwJ4T
      rw------ 1 www-data www-data 2.8G Jan 22 15:52 assignment_7oJUKn.tXoNJJ
      rw------ 1 www-data www-data 0 Jan 22 15:51 assignment_Ayq4Kh
      rw------ 1 www-data www-data 662M Jan 22 15:52 assignment_Ayq4Kh.Ar6CQZ
      rw------ 1 www-data www-data 71M Jan 22 15:52 assignment_Ayq4Kh.cargSW
      rw------ 1 www-data www-data 0 Jan 22 15:47 assignment_fRR4xX
      rw------ 1 www-data www-data 663M Jan 22 15:51 assignment_fRR4xX.6XQ2It
      rw------ 1 www-data www-data 742M Jan 22 15:52 assignment_fRR4xX.KAmfpb
      rw------ 1 www-data www-data 0 Jan 22 15:38 assignment_Ki518s
      rw------ 1 www-data www-data 669M Jan 22 15:47 assignment_Ki518s.RVRyBm
      rw------ 1 www-data www-data 354M Jan 22 15:50 assignment_Ki518s.XlU2B9
      rw------ 1 www-data www-data 0 Jan 22 15:29 assignment_wrkSjO
      rw------ 1 www-data www-data 647M Jan 22 15:30 assignment_wrkSjO.aMiyAs
      rw------ 1 www-data www-data 7.1G Jan 22 15:50 assignment_wrkSjO.wAsdQZ

      Two possible way to mitigate this:

      • add a check on the aggregate file size for all submissions to an assignment, and cancel the download if this exceeds a configurable limit (notifying the user to
      • set a flag in the user's session while a download is running, and update code to ignore further download requests if the link is clicked again.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              robhogg Rob Hardy
              Reporter:
              robhogg Rob Hardy
              Participants:
              Component watchers:
              Adrian Greeve, Jake Dallimore, Mathew May, Mihail Geshoski, Peter Dias, Sujith Haridasan, Matteo Scaramuccia, Andrew Lyons, Dongsheng Cai, Huong Nguyen, Jun Pataleta, Michael Hawkins, Shamim Rezaie, Simey Lameze
              Votes:
              2 Vote for this issue
              Watchers:
              5 Start watching this issue

                Dates

                Created:
                Updated: