Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-75295

Produce all mod_assign editpdf images from pages in a single shell call instead of per page

    XMLWordPrintable

Details

    • MOODLE_401_STABLE
    • MDL-75295_master
    • Hide

      Testing instructions are the same as MDL-70038 but running the task is slightly different because the task was changed from scheduled task to ad-hoc from 3.11 by MDL-71468

      Prerequisites

      • This test requires Ghostscript and Poppler to be installed on the server.
      • A PDF file is wanted, the larger it is the greater the difference in time that will be appreciated. (Feel free to use any of the attached PDFs on this issue)

      Setup

      1. Go to Site administration > Server > System paths page(admin/settings.php?section=systempaths).
      2. Verify that the Path to ghostscript setting points to your local gs path (be sure is correctly set).
      3. Verify that the Path to pdftoppm setting is empty.
      4. Verify the queue of conversions is empty (table task_adhoc does not have a record for \assignfeedback_editpdf\task\convert_submission) or run cron first to make sure existing ad-hoc tasks are removed from the task_adhoc table.
      5. Create a course.
      6. Add an assignment with following options:
        • Submission types: File submissions
        • Feedback types: Annotate PDF.

      Do the following tests before and after the patch

      Test - Ghostscript

      1. As a student, submit a PDF to the assignment.
      2. Verify the database table task_adhoc has 1 record for \assignfeedback_editpdf\task\convert_submission.
      3. Run the scheduled task:

        time php admin/cli/adhoc_task.php --execute

        • This execution will use gs to generate PNG files from your PDF file. Record the total execution time.
      4. (After the patch only)
        1. As admin or teacher, go to grade the submission.
        2. Confirm you see the submission content. (The converted PDF is shown on the assignment grading interface)

      Test - Poppler

      1. Set up the path for the pdftoppm tool:
        • Go to Site administration > Server > System paths page(admin/settings.php?section=systempaths).
        • Add your local pdftoppm path to the Path to pdftoppm setting (be sure it is correctly set).
      2. Log in as another student
      3. Submit the same file
      4. Run the scheduled task:

        time php admin/cli/adhoc_task.php --execute

        • This execution will use gs to generate PNG files from your PDF file. Record the total execution time.
      5. (After the patch only)
        1. As admin or teacher, go to grade the submission.
        2. Confirm you see the submission content. (The converted PDF is shown on the assignment grading interface)

      Confirmation

      1. Confirm that the processing times in both Ghostscript and Poppler after the patch are lower than the times before the patch.
      Show
      Testing instructions are the same as  MDL-70038 but running the task is slightly different because the task was changed from scheduled task to ad-hoc from 3.11 by MDL-71468 Prerequisites This test requires Ghostscript and Poppler to be installed on the server. A PDF file is wanted, the larger it is the greater the difference in time that will be appreciated. (Feel free to use any of the attached PDFs on this issue) Setup Go to Site administration > Server > System paths page(admin/settings.php?section=systempaths). Verify that the Path to ghostscript setting points to your local gs path (be sure is correctly set). Verify that the Path to pdftoppm setting is empty. Verify the queue of conversions is empty (table task_adhoc does not have a record for \assignfeedback_editpdf\task\convert_submission ) or run cron first to make sure existing ad-hoc tasks are removed from the task_adhoc table. Create a course. Add an assignment with following options: Submission types: File submissions Feedback types: Annotate PDF. Do the following tests before and after the patch Test - Ghostscript As a student, submit a PDF to the assignment. Verify the database table task_adhoc has 1 record for \assignfeedback_editpdf\task\convert_submission . Run the scheduled task: time php admin/cli/adhoc_task.php --execute This execution will use gs to generate PNG files from your PDF file. Record the total execution time. (After the patch only) As admin or teacher, go to grade the submission. Confirm you see the submission content. (The converted PDF is shown on the assignment grading interface) Test - Poppler Set up the path for the pdftoppm tool: Go to Site administration > Server > System paths page(admin/settings.php?section=systempaths). Add your local pdftoppm path to the Path to pdftoppm setting (be sure it is correctly set). Log in as another student Submit the same file Run the scheduled task: time php admin/cli/adhoc_task.php --execute This execution will use gs to generate PNG files from your PDF file. Record the total execution time. (After the patch only) As admin or teacher, go to grade the submission. Confirm you see the submission content. (The converted PDF is shown on the assignment grading interface) Confirmation Confirm that the processing times in both Ghostscript and Poppler after the patch are lower than the times before the patch.

    Description

      I suspect we can vastly improve the speed of the image conversion done in MDL-70038 by unrolling this loop and have a cli command that parses the pdf once and produces N images for all the pages instead of shell commands for each page:

      https://github.com/moodle/moodle/blob/master/mod/assign/feedback/editpdf/classes/document_services.php#L441-L443

       

      Attachments

        1. 10840.pdf
          24.76 MB
        2. A17_FlightPlan.pdf
          19.74 MB
        3. MDL-75295_after_Ghostscript_1.png
          MDL-75295_after_Ghostscript_1.png
          118 kB
        4. MDL-75295_after_Ghostscript_2.png
          MDL-75295_after_Ghostscript_2.png
          104 kB
        5. MDL-75295_after_Ghostscript_3.png
          MDL-75295_after_Ghostscript_3.png
          377 kB
        6. MDL-75295_after_Poppler_1.png
          MDL-75295_after_Poppler_1.png
          61 kB
        7. MDL-75295_after_Poppler_2.png
          MDL-75295_after_Poppler_2.png
          354 kB
        8. MDL-75295_before_Ghostscript.png
          MDL-75295_before_Ghostscript.png
          63 kB
        9. MDL-75295_before_Poppler.png
          MDL-75295_before_Poppler.png
          58 kB
        10. Test.pdf
          1018 kB

        Issue Links

          Activity

            People

              tomotsuyuki Tomo Tsuyuki
              brendanheywood Brendan Heywood
              Brendan Heywood Brendan Heywood
              Jun Pataleta Jun Pataleta
              Angelia Dela Cruz Angelia Dela Cruz
              Votes:
              2 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 6 hours
                  6h

                  Clockify

                    Error rendering 'clockify-timesheets-time-tracking-reports:timer-sidebar'. Please contact your Jira administrators.