Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-75448

Annotate PDF fails to convert rotated images when using polling converter

XMLWordPrintable

    • MOODLE_401_STABLE
    • MOODLE_311_STABLE, MOODLE_400_STABLE
    • MDL-75448-master
    • Hide

      Setup

      1. Install Dummy Document Converter plugin:
        git clone git@github.com:catalyst/moodle-fileconverter_dummy.git files/converter/dummy/
      2. Run the upgrade CLI:
        php admin/cli/upgrade.php
      3. Enable Dummy Document Converter plugin:
        php admin/cli/cfg.php --name='converter_plugins_sortorder' --set='dummy'
      4. Purge caches:
        php admin/cli/purge_caches.php
      5. Create two students (Student One and Student Two).
      6. Create a course.
      7. Enrol both students to the course.
      8. Create an assignment activity with Submission types set to File submissions and Annotate PDF enabled in Feedback types.
      9. Log in Student One.
      10. Access the assignment activity and submit the IMG-5055.JPG file (attached to the tracker) as a submission.

      Replicate the issue (before the patch is applied):

      1. Run the task: 

        M4.1: php admin/cli/adhoc_task.php --execute
        		 
        Older versions: php admin/cli/scheduled_task.php --execute='\assignfeedback_editpdf\task\convert_submissions'

      2. Confirm, that the task completes.
      3. Run the SQL and confirm, that a conversion record has been created and it has a status of 1 (STATUS_IN_PROGRESS): 

        moodle=# select * from mdl_file_conversion order by id desc limit 3;
        		 
         id | usermodified | timecreated | timemodified | sourcefileid | targetformat | status | statusmessage |           converter            | destfileid | data 
        ----+--------------+-------------+--------------+--------------+--------------+--------+---------------+--------------------------------+------------+------
        		 
          1 |            2 |  1660529800 |   1660529800 |           29 | pdf          |      1 | Oh henlo      | \fileconverter_dummy\converter |         31 | 
        (1 row)

      4. Confirm, that the file record with sourcefileid exists: 

        moodle=# select id, contenthash, filearea, filename from mdl_files where id = 29;
        		 
         id |               contenthash                |    filearea     |   filename   
        ----+------------------------------------------+-----------------+--------------
        		 
         29 | 28baa8fa004cc4349db7e7892f3e839c6d0c6e5d | tmp_rotated_jpg | IMG-5055.JPG
        		 
        (1 row)

      5. Re-run the task: 

        M4.1: php admin/cli/adhoc_task.php --execute
        		 
        Older versions: php admin/cli/scheduled_task.php --execute='\assignfeedback_editpdf\task\convert_submissions'

      6. Confirm, that new conversion has been created: 

        moodle=# select * from mdl_file_conversion order by id desc limit 3;
        		 
         id | usermodified | timecreated | timemodified | sourcefileid | targetformat | status | statusmessage |           converter            | destfileid | data 
        ----+--------------+-------------+--------------+--------------+--------------+--------+---------------+--------------------------------+------------+------
        		 
          2 |            2 |  1660532470 |   1660532470 |           34 | pdf          |      1 | Oh henlo      | \fileconverter_dummy\converter |         35 | 
          1 |            2 |  1660529800 |   1660529800 |           29 | pdf          |      1 | Oh henlo      | \fileconverter_dummy\converter |         31 | 
        (2 rows)

      7. Confirm, that old file record has been removed and the new one has been created: 

        moodle=# select id, contenthash, filearea, filename from mdl_files where id in (29, 34);
        		 
         id |               contenthash                |    filearea     |   filename   
        ----+------------------------------------------+-----------------+--------------
        		 
         34 | 28baa8fa004cc4349db7e7892f3e839c6d0c6e5d | tmp_rotated_jpg | IMG-5055.JPG
        		 
        (1 row)

      8. Outcome: Re-running the task deletes previous file record and creates new conversion.

      Testing instructions:

      1. Apply the patch.
      2. Re-run the task: 

        M4.1: php admin/cli/adhoc_task.php --execute
        		 
        Older versions: php admin/cli/scheduled_task.php --execute='\assignfeedback_editpdf\task\convert_submissions'

      3. Confirm, that the task completes.
      4. Run the following SQL: 

        moodle=# select * from mdl_file_conversion order by id desc;
        		 
         id | usermodified | timecreated | timemodified | sourcefileid | targetformat | status | statusmessage |           converter            | destfileid | data 
        ----+--------------+-------------+--------------+--------------+--------------+--------+---------------+--------------------------------+------------+------
        		 
          4 |            2 |  1660536690 |   1660537317 |           38 | pdf          |      2 | Oh henlo      | \fileconverter_dummy\converter |         39 | 
        (1 row)

      5. Confirm, that the latest conversion has a status of 2 (STATUS_COMPLETE).
      6. Log in as Student Two.
      7. Access the assignment activity and submit the IMG-5056.JPG file (attached to the tracker) as a submission.
      8. Run the task twice with at least 60 interval: 

        M4.1: php admin/cli/adhoc_task.php --execute
        		 
        Older versions: php admin/cli/scheduled_task.php --execute='\assignfeedback_editpdf\task\convert_submissions'

      9. Run the following SQL: 

        moodle=# select * from mdl_file_conversion order by id desc;
        		 
         id | usermodified | timecreated | timemodified | sourcefileid | targetformat | status | statusmessage |           converter            | destfileid | data 
        ----+--------------+-------------+--------------+--------------+--------------+--------+---------------+--------------------------------+------------+------
        		 
          5 |            2 |  1660536792 |   1660537437 |           42 | pdf          |      2 | Oh henlo      | \fileconverter_dummy\converter |         43 | 
          4 |            2 |  1660536690 |   1660537317 |           38 | pdf          |      2 | Oh henlo      | \fileconverter_dummy\converter |         39 | 
        (2 rows)

      10. Confirm, that the latest conversion has a status of 2 (STATUS_COMPLETE).
      Show
      Setup Install  Dummy Document Converter  plugin: git clone git@github.com:catalyst/moodle-fileconverter_dummy.git files/converter/dummy/ Run the upgrade CLI: php admin/cli/upgrade.php Enable  Dummy Document Converter  plugin: php admin/cli/cfg.php --name='converter_plugins_sortorder' --set='dummy' Purge caches: php admin/cli/purge_caches.php Create two students (Student One and Student Two). Create a course. Enrol both students to the course. Create an assignment activity with Submission types  set to  File submissions  and  Annotate PDF  enabled in  Feedback types . Log in Student One. Access the assignment activity and submit the  IMG-5055.JPG  file (attached to the tracker) as a submission. Replicate the issue (before the patch is applied): Run the task: M4. 1 : php admin/cli/adhoc_task.php --execute Older versions: php admin/cli/scheduled_task.php --execute= '\assignfeedback_editpdf\task\convert_submissions' Confirm , that the task completes. Run the SQL and confirm , that a conversion record has been created and it has a status of 1 ( STATUS_IN_PROGRESS ): moodle=# select * from mdl_file_conversion order by id desc limit 3 ; id | usermodified | timecreated | timemodified | sourcefileid | targetformat | status | statusmessage | converter | destfileid | data ----+--------------+-------------+--------------+--------------+--------------+--------+---------------+--------------------------------+------------+------ 1 | 2 | 1660529800 | 1660529800 | 29 | pdf | 1 | Oh henlo | \fileconverter_dummy\converter | 31 | ( 1 row) Confirm , that the file record with sourcefileid exists: moodle=# select id, contenthash, filearea, filename from mdl_files where id = 29 ; id | contenthash | filearea | filename ----+------------------------------------------+-----------------+-------------- 29 | 28baa8fa004cc4349db7e7892f3e839c6d0c6e5d | tmp_rotated_jpg | IMG- 5055 .JPG ( 1 row) Re-run the task: M4. 1 : php admin/cli/adhoc_task.php --execute Older versions: php admin/cli/scheduled_task.php --execute= '\assignfeedback_editpdf\task\convert_submissions' Confirm , that new conversion has been created: moodle=# select * from mdl_file_conversion order by id desc limit 3 ; id | usermodified | timecreated | timemodified | sourcefileid | targetformat | status | statusmessage | converter | destfileid | data ----+--------------+-------------+--------------+--------------+--------------+--------+---------------+--------------------------------+------------+------ 2 | 2 | 1660532470 | 1660532470 | 34 | pdf | 1 | Oh henlo | \fileconverter_dummy\converter | 35 | 1 | 2 | 1660529800 | 1660529800 | 29 | pdf | 1 | Oh henlo | \fileconverter_dummy\converter | 31 | ( 2 rows) Confirm , that old file record has been removed and the new one has been created: moodle=# select id, contenthash, filearea, filename from mdl_files where id in ( 29 , 34 ); id | contenthash | filearea | filename ----+------------------------------------------+-----------------+-------------- 34 | 28baa8fa004cc4349db7e7892f3e839c6d0c6e5d | tmp_rotated_jpg | IMG- 5055 .JPG ( 1 row) Outcome: Re-running the task deletes previous file record and creates new conversion. Testing instructions: Apply the patch. Re-run the task: M4. 1 : php admin/cli/adhoc_task.php --execute Older versions: php admin/cli/scheduled_task.php --execute= '\assignfeedback_editpdf\task\convert_submissions' Confirm , that the task completes. Run the following SQL: moodle=# select * from mdl_file_conversion order by id desc; id | usermodified | timecreated | timemodified | sourcefileid | targetformat | status | statusmessage | converter | destfileid | data ----+--------------+-------------+--------------+--------------+--------------+--------+---------------+--------------------------------+------------+------ 4 | 2 | 1660536690 | 1660537317 | 38 | pdf | 2 | Oh henlo | \fileconverter_dummy\converter | 39 | ( 1 row) Confirm , that the latest conversion has a status of 2 ( STATUS_COMPLETE ). Log in as Student Two. Access the assignment activity and submit the IMG-5056.JPG file (attached to the tracker) as a submission. Run the task twice with at least 60 interval: M4. 1 : php admin/cli/adhoc_task.php --execute Older versions: php admin/cli/scheduled_task.php --execute= '\assignfeedback_editpdf\task\convert_submissions' Run the following SQL: moodle=# select * from mdl_file_conversion order by id desc; id | usermodified | timecreated | timemodified | sourcefileid | targetformat | status | statusmessage | converter | destfileid | data ----+--------------+-------------+--------------+--------------+--------------+--------+---------------+--------------------------------+------------+------ 5 | 2 | 1660536792 | 1660537437 | 42 | pdf | 2 | Oh henlo | \fileconverter_dummy\converter | 43 | 4 | 2 | 1660536690 | 1660537317 | 38 | pdf | 2 | Oh henlo | \fileconverter_dummy\converter | 39 | ( 2 rows) Confirm , that the latest conversion has a status of 2 ( STATUS_COMPLETE ).

      After MDL-71468 has been integrated we're having some assignfeedback_editpdf\task\convert_submission adhoc task constantly failing with message Document conversion completed with status 0.

      After having a closer look it's been found that when using polling converter (that creates a conversion on start_document_conversion() and requires poll_conversion_status() to be called to retrieve the converted document) Annotate PDF fails to convert images that get rotated before conversion.

      It rotates the image, creates a new file (rotated image), then sends new file to converter. When it runs again, intead of polling exising converted file it rotates the image again, creates new file and sends new file to the converter. See https://github.com/moodle/moodle/blob/896e126a667c5cb805c06d5cd6e5f6263f94bd27/mod/assign/feedback/editpdf/classes/document_services.php#L200-L203

      With Unoconv it works as expected as the actual conversion is happening in start_document_conversion() and the method returns the converted file (polling is not used).

      As I can see exisinting file should be re-used here https://github.com/moodle/moodle/blob/896e126a667c5cb805c06d5cd6e5f6263f94bd27/mod/assign/feedback/editpdf/classes/document_services.php#L916-L924

      Otherwise, it creates new mdl_files and mdl_file_conversion records every time assignfeedback_editpdf\task\convert_submission tries to get the combined document.

        1. IMG-5055.JPG
          IMG-5055.JPG
          1.32 MB
        2. IMG-5056.JPG
          IMG-5056.JPG
          1.28 MB
        3. MDL-75448_master_after_1.png
          MDL-75448_master_after_1.png
          22 kB
        4. MDL-75448_master_after_2.png
          MDL-75448_master_after_2.png
          27 kB
        5. MDL-75448_master_before_1.png
          MDL-75448_master_before_1.png
          87 kB
        6. MDL-75448_master_before_2.png
          MDL-75448_master_before_2.png
          38 kB
        7. MDL-75448_master_before_3.png
          MDL-75448_master_before_3.png
          38 kB
        8. MDL-75448_v311_after_1.png
          MDL-75448_v311_after_1.png
          22 kB
        9. MDL-75448_v311_after_2.png
          MDL-75448_v311_after_2.png
          27 kB
        10. MDL-75448_v311_before_1.png
          MDL-75448_v311_before_1.png
          46 kB
        11. MDL-75448_v311_before_2.png
          MDL-75448_v311_before_2.png
          13 kB
        12. MDL-75448_v311_before_3.png
          MDL-75448_v311_before_3.png
          27 kB
        13. MDL-75448_v311_before_4.png
          MDL-75448_v311_before_4.png
          23 kB
        14. MDL-75448_v311_before_5.png
          MDL-75448_v311_before_5.png
          17 kB
        15. MDL-75448_v400_after_1.png
          MDL-75448_v400_after_1.png
          22 kB
        16. MDL-75448_v400_after_2.png
          MDL-75448_v400_after_2.png
          27 kB
        17. MDL-75448_v400_before_1.png
          MDL-75448_v400_before_1.png
          46 kB
        18. MDL-75448_v400_before_2.png
          MDL-75448_v400_before_2.png
          40 kB
        19. MDL-75448_v400_before_3.png
          MDL-75448_v400_before_3.png
          39 kB

            mikhailgolenkov Misha Golenkov
            mikhailgolenkov Misha Golenkov
            cameron1729 cameron1729
            Sara Arjona (@sarjona) Sara Arjona (@sarjona)
            Angelia Dela Cruz Angelia Dela Cruz
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved:

                Estimated:
                Original Estimate - 0 minutes
                0m
                Remaining:
                Remaining Estimate - 0 minutes
                0m
                Logged:
                Time Spent - 3 hours, 10 minutes
                3h 10m

                  Error rendering 'clockify-timesheets-time-tracking-reports:timer-sidebar'. Please contact your Jira administrators.