Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-68533

Allow mod_assign download all assignments to be streamed

XMLWordPrintable

    • MOODLE_311_STABLE
    • MDL-68533_master-clean_param
    • Hide
      Increase file upload size limit
      1. Edit your php.ini file. e.g.

        sudo xed /etc/php/7.4/apache2/php.ini
        

      2. Set the following config settings to the following values:
        • upload_max_filesize = 200M
        • post_max_size = 200M
      3. Save the php.ini file and restart apache

        sudo service apache2 restart
        

      4. Go to "Site administration / Security / Site security settings" and set "Maximum uploaded file size" to "Site upload limit (200MB)"
      5. Save the changes
      Test
      1. Log in as admin/course creator
      2. Create a course with a student.
      3. Create an assignment activity.
      4. On another browser session log in a student.
      5. Add a submission.
      6. Upload several files with a total size > 100 MB.
      7. On the admin's/course creator's browser session, go to the assignment and view all submissions.
      8. Select the student's submission and set "With selected..." to "Download selected submissions" then press "Go"
      9. Confirm the download of the submission.
      10. Confirm, the archive is downloading.
      11. While the archive is downloading open a new tab with any Moodle page and confirm it can be open, which means the session is not locked during assignment download.
      12. Confirm, that archive content is correct when finished downloading.
      Show
      Increase file upload size limit Edit your php.ini file. e.g. sudo xed /etc/php/7.4/apache2/php.ini Set the following config settings to the following values: upload_max_filesize = 200M post_max_size = 200M Save the php.ini file and restart apache sudo service apache2 restart Go to " Site administration / Security / Site security settings " and set " Maximum uploaded file size " to " Site upload limit (200MB) " Save the changes Test Log in as admin/course creator Create a course with a student. Create an assignment activity. On another browser session log in a student. Add a submission. Upload several files with a total size > 100 MB. On the admin's/course creator's browser session, go to the assignment and view all submissions. Select the student's submission and set " With selected... " to " Download selected submissions " then press " Go " Confirm the download of the submission. Confirm , the archive is downloading. While the archive is downloading open a new tab with any Moodle page and confirm it can be open, which means the session is not locked during assignment download. Confirm , that archive content is correct when finished downloading.

      This is almost the same pattern as MDL-68481, broadly the issue is this synchronously zips and download s massive chunk of files which can take ages and either hit or force various timeouts to be increased.

      I'm proposing:

      1) the zip preparation is done as a first step with a progress bar which then redirects to the second step which downloads. As the download is in a second http request it would need to copy the zip into shared disk and then clean it up

      2) both steps above can close the session

      3) bonus points (might be contentious): every time an assignment is uploaded then an ad hoc task can be rescheduled / queued for some healthy point into the future, say as 6 hours, which does the zip step ahead of time and stores it in the file api. By choosing 6 hours then many constant assignment uploads keep pushing back the final zip until it quiets down which should happen on the due date, lets say at midnight, and then the next day (assuming no late submissions) the zip is ready to go for the markers to download:

      https://docs.moodle.org/dev/Task_API#Rescheduling_an_adhoc_task

      If anyone still wants it in the mean time then you redo step 1 synchronously, and when it is done it would also de-queue the ad hoc task as it isn't needed any more. Most assignment files will not compress well because they are already compressed, so if this gets implemented maybe it could be behind a setting as it is trading off disk space vs download and generation time. If this setting is on then it would NOT need to cleanup the files in step 1, but we may want to clean up old zip files from old assignments so reduce disk.

            markn Mark Nelson
            brendanheywood Brendan Heywood
            Brendan Heywood Brendan Heywood
            Jun Pataleta Jun Pataleta
            Janelle Barcega Janelle Barcega
            Votes:
            4 Vote for this issue
            Watchers:
            14 Start watching this issue

              Created:
              Updated:
              Resolved:

                Estimated:
                Original Estimate - 0 minutes
                0m
                Remaining:
                Remaining Estimate - 0 minutes
                0m
                Logged:
                Time Spent - 3 hours, 40 minutes
                3h 40m

                  Error rendering 'clockify-timesheets-time-tracking-reports:timer-sidebar'. Please contact your Jira administrators.