Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-66107

Stop bundling all files in first.js

XMLWordPrintable

    • MOODLE_38_STABLE, MOODLE_401_STABLE, MOODLE_403_STABLE
    • MDL-66107-master
    • Hide

      Behat will test this issue for regressions. Most of the other testing will happen by use, during the QA cycle, and any other smoke testing.

      This is a really difficult one to test because it's really a matter of weighing up whether we are happy with the change or not.

      To compare, I would suggest:

      1. Ensure that you are using the JS cache in Moodle (mdk run undev or ensure that the cachejs option is enabled)
      2. Open developer tools to the "Network" tab
      3. Use the Throttling tools to compare different connection speeds
      4. Repeat page fetches multiple times, ensuring that you clear the cache between each time (or disable the cache in the browser during the tests)

      Things we expect as a result of this change:

      1. An increase in the quantity of requests per page load
      2. A decrease in the volume of data transmitted over the wire
      3. An end to the content of first.js being duplicated in another request (typically event.js)
      4. When navigating to differnet pages, only those AMD modules which have not yet been fetched will be fetched

      Note: On a really slow connection, this may be seen to have a performance impact, but this is arguably not the case.
      At the moment we are forcing the client to download a large (500kB) file once, or twice. This may take 20-30 seconds.
      The change here will mean that clients instead fetch a larger number of smaller files (for example, around 1-3kB each).
      Because there are many of them, and browsers will only perform a limited number of requests to the same server at once (typically 7 from memory), these will happen is small batches.
      Overall less data will be transmitted, and likely in a similar time period (on a slower connection), but there will be more content transferred.

      It's also worth noting that this usually only affects the first page visit to the site. After this, the core componentry has been loaded and will remain in cache.

      Other testing of this change

      1. Ensure that you are using the JS cache in Moodle (mdk run undev or ensure that the cachejs option is enabled)
      2. Open your browser developer tools to Moodle
      3. Try and open a Moodle JS file, for example lib/amd/src/str.js
        1. Confirm you are not able to find a source file version - only the minified variant*
      4. In your config.php file set: 

        $CFG->productionsourcemaps = true;

      5. Purge caches
      6. Refresh the page
      7. Try and open a Moodle JS file, for example lib/amd/src/str.js
        1. Confirm you are now able to view a source-mapped version of the content
      Show
      Behat will test this issue for regressions. Most of the other testing will happen by use, during the QA cycle, and any other smoke testing. This is a really difficult one to test because it's really a matter of weighing up whether we are happy with the change or not. To compare, I would suggest: Ensure that you are using the JS cache in Moodle ( mdk run undev or ensure that the cachejs option is enabled) Open developer tools to the "Network" tab Use the Throttling tools to compare different connection speeds Repeat page fetches multiple times, ensuring that you clear the cache between each time (or disable the cache in the browser during the tests) Things we expect as a result of this change: An increase in the quantity of requests per page load A decrease in the volume of data transmitted over the wire An end to the content of first.js being duplicated in another request (typically event.js ) When navigating to differnet pages, only those AMD modules which have not yet been fetched will be fetched Note : On a really slow connection, this may be seen to have a performance impact, but this is arguably not the case. At the moment we are forcing the client to download a large (500kB) file once, or twice. This may take 20-30 seconds. The change here will mean that clients instead fetch a larger number of smaller files (for example, around 1-3kB each). Because there are many of them, and browsers will only perform a limited number of requests to the same server at once (typically 7 from memory), these will happen is small batches. Overall less data will be transmitted, and likely in a similar time period (on a slower connection), but there will be more content transferred. It's also worth noting that this usually only affects the first page visit to the site. After this, the core componentry has been loaded and will remain in cache. Other testing of this change Ensure that you are using the JS cache in Moodle ( mdk run undev or ensure that the cachejs option is enabled) Open your browser developer tools to Moodle Try and open a Moodle JS file, for example lib/amd/src/str.js Confirm you are not able to find a source file version - only the minified variant * In your config.php file set: $CFG->productionsourcemaps = true; Purge caches Refresh the page Try and open a Moodle JS file, for example lib/amd/src/str.js Confirm you are now able to view a source-mapped version of the content
    • 6
    • Team Hedgehog 4.1 sprint 0.4B, Team Hedgehog 4.1 sprint 0 rev, Team Hedgehog 4.1 pre 1.1

      I've done a bit of research, and I believe that bundling all compiled JS in first.js is a false economy with significant drawbacks. These drawback include:

      1. At the moment, the first time a page is loaded we load all AMD JS in Moodle.
      2. This currently accounts for approximately 900KB (2019), and growing (2.1MB uncompressed and 461kB over the wire in 2022).
      3. Transfer time is also high with my 2022 development machine taking a significant amount of time to transfer the wire localy (Lots of variation but I had values in ms of: 742, 132, 148, 849, 829. 1050).
      4. The TTFB is also high at about 80ms as we combine all of the various files and then echo them out at the end. This therefore also has an impact on memory as all of this content is stored in memory until it is echoed out (peak memory usage just before we finish compiling is about 1.3MB).
      5. If any single JS file has an issue then all JS on the site will fail.

      All of this means negative things for scale, for user experience, reliability, for browser memory usage, and for debugging.

      From what I can see, whilst the initial page load will require more files to be fetched, these will be fetched faster, and cached more efficiently. We will also be more reliable (JS errors not killing the page), and less browser memory.

      Additional changes of note

      Source maps

      At present, the differences between production and development include:

      • combo-load vs. individual load
      • no sourcemap in prod vs. sourcemap in dev

      As we are removing support for combo-loading, I feel it is also prudent to add the missing sourcemap support in production code.
      This was not originally included due to a bug in Firefox which has been fixed for several years. This bug meant that we had to include the entire source content in the source map, rather than referencing the original file. In the case of the combo-loader, this meant a very large sourceContent in the single source map, and the sourcemap did not work in Firefox well as a result.

      Now that this issue has been resolved in Firefox, we can remove the sourceContent from the sourcemap, and include it in production code.

      Legacy define

      In MDL-62497 we included some changes to help developers transition to using the new transpilation process. These features are no longer relevant and it is safe to remove them.

      Summary

      Each part of this change is well documented in the individual commit messages with explanation of the changes.

      The key change here is to stop using a combo loader to serve all JS content in a single (very large) file, and serve individual modules on request.

      The main concern with this change is a change from a larger over-the-wire file-size to lots of smaller individual requets made to the server.

      In my local testing, it seems that these changes are beneficial in most situations – the main area of concern being slower connections where round-trip time is much higher as there are more individual connections. These concerns are offset by the massive reduction in transferred filesize as only the requested modules are now fetched.

        1. image.png
          image.png
          199 kB
        2. image-2022-10-31-18-06-03-823.png
          image-2022-10-31-18-06-03-823.png
          31 kB

            Unassigned Unassigned
            dobedobedoh Andrew Lyons
            Stevani Andolo Stevani Andolo
            Ilya Tregubov Ilya Tregubov
            John Edward Pedregosa John Edward Pedregosa
            Votes:
            14 Vote for this issue
            Watchers:
            29 Start watching this issue

              Created:
              Updated:

                Estimated:
                Original Estimate - 0 minutes
                0m
                Remaining:
                Remaining Estimate - 0 minutes
                0m
                Logged:
                Time Spent - 2 hours, 46 minutes
                2h 46m

                  Error rendering 'clockify-timesheets-time-tracking-reports:timer-sidebar'. Please contact your Jira administrators.