Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-69207

Add library file caching to h5p

    XMLWordPrintable

Details

    • MOODLE_38_STABLE, MOODLE_39_STABLE
    • MOODLE_310_STABLE
    • MDL-69207-master
    • Hide

      Before installing the patch

      We emulate remote file storage, adding a delay when retrieving the content of the files:
      1. Download the attached file fakeremotefilestorage.diff in your Moodle root folder.
      2. Apply the patch:

      patch -p1 < fakeremotefilestorage.diff

      3. Log in as admin.
      4. Go to 'Content bank' (in the drawer for Boost theme, in Navigation block > Site pages for Classic theme).
      5. Open DevTools by pressing Control+Shift+J or Command+Option+J (Mac).
      6. Click on the Network tab.
      7. Type embed.php? into the Filter text box.
      8. Click on 'Upload' button.
      9. Upload attached interactivevideo.h5p file and 'Save changes'.
      10. Wait until the page displays the H5P content.
      11. Write down the load time of the embed.php request.
      12. Click on the Close button.
      13. Repeat steps 8-11: the timing should be similar.

      After installing the patch and upgrading the site

      14. Go to 'Content bank' (in the drawer for Boost theme, in Navigation block > Site pages for Classic theme).
      15. Repeat steps 8-11: the cache is filled with the H5P library files.
      16. Repeat steps 8-11: H5P content display should be faster.
      17. Confirm that the load time in step 16 is significantly less than the previous ones.

      Show
      Before installing the patch We emulate remote file storage, adding a delay when retrieving the content of the files: 1. Download the attached file fakeremotefilestorage.diff in your Moodle root folder. 2. Apply the patch: patch -p1 < fakeremotefilestorage. diff 3. Log in as admin. 4. Go to 'Content bank' (in the drawer for Boost theme, in Navigation block > Site pages for Classic theme). 5. Open DevTools by pressing Control+Shift+J or Command+Option+J (Mac). 6. Click on the Network tab. 7. Type embed.php? into the Filter text box. 8. Click on 'Upload' button. 9. Upload attached interactivevideo.h5p file and 'Save changes'. 10. Wait until the page displays the H5P content. 11. Write down the load time of the embed.php request. 12. Click on the Close button. 13. Repeat steps 8-11: the timing should be similar. After installing the patch and upgrading the site 14. Go to 'Content bank' (in the drawer for Boost theme, in Navigation block > Site pages for Classic theme). 15. Repeat steps 8-11: the cache is filled with the H5P library files. 16. Repeat steps 8-11: H5P content display should be faster. 17. Confirm that the load time in step 16 is significantly less than the previous ones.

    Description

      When adding an h5p activity, Moodle downloads around 1500 files from the h5p library filearea in order to generate a zip file which then gets pushed back into the file store.

      With an alternate file storage provider like Catalyst's ObjectFS backed by a remote store like AWS S3, downloading these library files one by one can be an extremely slow task - around 2 minutes to fetch 1500 files with the additional s3 api overhead.

      It would be nice to have these files available in a muc cache.

      Storing all library files currently available comes out to around 32mb (deduplicated by contenthash) on disk or just over 2mb in an igbinary+gzip redis store and I was able to bring down the activity creation time to 2 seconds.

      We already have this in production and is working very well.

      Attachments

        1. after.png
          after.png
          84 kB
        2. before.png
          before.png
          88 kB
        3. fakeremotefilestorage.diff
          0.7 kB
        4. interactivevideo.h5p
          1.99 MB
        5. Test_AfterPatch_FirstUpload.png
          Test_AfterPatch_FirstUpload.png
          238 kB
        6. Test_AfterPatch_NextUpload.png
          Test_AfterPatch_NextUpload.png
          200 kB
        7. Test_BeforePatch_FirstUpload.png
          Test_BeforePatch_FirstUpload.png
          156 kB
        8. Test_BeforePatch_NextUpload.png
          Test_BeforePatch_NextUpload.png
          181 kB

        Activity

          People

            jtomkinson Jordan Tomkinson
            jtomkinson Jordan Tomkinson
            cameron1729 cameron1729
            Víctor Déniz Falcón Víctor Déniz Falcón
            Carlos Escobedo Carlos Escobedo
            Matteo Scaramuccia, Amaia Anabitarte, Carlos Escobedo, Ferran Recio, Ilya Tregubov, Sara Arjona (@sarjona), Amaia Anabitarte, Carlos Escobedo, Ferran Recio, Ilya Tregubov, Sara Arjona (@sarjona)
            Votes:
            7 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:
              9/Nov/20

              Time Tracking

                Estimated:
                Original Estimate - 0 minutes
                0m
                Remaining:
                Remaining Estimate - 0 minutes
                0m
                Logged:
                Time Spent - 2 days, 1 hour, 10 minutes
                2d 1h 10m