Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-67449

Grunt amd:dist very slow

    XMLWordPrintable

Details

    • MOODLE_38_STABLE
    • MOODLE_39_STABLE
    • MDL-67449-master
    • Hide

      Build all from root directory

      1. Change directory into your moodle root
      2. Run grunt:

        grunt
        

      3. Confirm there were no files changes:

        git status
        

      Watch from root directory

      1. Change directory into your moodle root
      2. Run grunt watch:

        grunt watch
        

      3. Open a second terminal
      4. Touch some files. For each file confirm that the file builds.

        touch mod/forum/amd/src/discussion.js
        touch mod/forum/amd/src/local/layouts.js
        touch mod/forum/amd/src/local/layout/fullscreen.js
        touch mod/assign/yui/src/history/js/history.js
        touch mod/forum/styles.css
        touch theme/boost/scss/moodle/*.scss
        touch theme/boost/scss/moodle/editor.scss
        

      5. Touch some more files and ensure that thy do not trigger a build:

        touch theme/boost/style/*.css
        

        1. Confirm that there were no changes:

          git status
          

      Build all from component directory

      1. Change directory into mod/assign:

        cd mod/assign
        

      2. Run grunt:

        grunt
        

      3. Confirm there were no files changes:

        git status
        

      Watch from component directory

      1. Change directory into mod/assign:

        cd mod/assign
        

      2. Run grunt watch:

        grunt watch
        

      3. Open a second terminal
      4. Touch some files. For each file confirm that the file builds.

        touch mod/assign/yui/src/history/js/history.js
        touch mod/assign/styles.css
        touch mod/assign/amd/src/grading_panel.js
        

      5. Touch some more files and ensure that thy do not trigger a build:

        touch mod/forum/amd/src/discussion.js
        touch mod/forum/amd/src/local/layouts.js
        touch mod/forum/amd/src/local/layout/fullscreen.js
        touch theme/boost/style/*.css
        touch theme/boost/scss/moodle/*.scss
        touch theme/boost/scss/moodle/editor.scss
        touch mod/forum/styles.css
        

        1. Confirm that there were no changes:

          git status
          

      Watch from a different component directory

      1. Change directory into mod/forum:

        cd mod/forum
        

      2. Run grunt watch:

        grunt watch
        

      3. Open a second terminal
      4. Touch some files. For each file confirm that the file builds.

        touch mod/forum/amd/src/discussion.js
        touch mod/forum/amd/src/local/layouts.js
        touch mod/forum/amd/src/local/layout/fullscreen.js
        touch mod/forum/styles.css
        

      5. Touch some more files and ensure that thy do not trigger a build:

        touch mod/assign/yui/src/history/js/history.js
        touch mod/assign/styles.css
        touch mod/assign/amd/src/grading_panel.js
        touch theme/boost/style/*.css
        touch theme/boost/scss/moodle/*.scss
        touch theme/boost/scss/moodle/editor.scss
        

        1. Confirm that there were no changes:

          git status
          

      Watch and build from a theme directory

      1. Change directory into theme/boost

        cd theme/boost
        

      2. Run grunt:

        grunt
        

      3. Confirm there were no files changes:

        git status
        

      4. Run grunt watch
      5. Open a second terminal
      6. Touch some files. For each file confirm that the file builds.

        touch theme/boost/scss/moodle/*.scss
        touch theme/boost/scss/moodle/editor.scss
        

      7. Touch some more files and ensure that thy do not trigger a build:

        touch theme/boost/style/*.css
        

        1. Confirm that there were no changes:

          git status
          

      Watch and build from an AMD directory

      1. Change directory into mod/forum/amd:

        cd mod/forum/amd
        

      2. Run grunt:

        grunt
        

        1. Confirm that only the startup, eslint:amd, and babel:dist tasks ran
      3. Run grunt watch
      4. Open a second terminal
      5. Touch some files. For each file confirm that the file builds.

        touch mod/forum/amd/src/discussion.js
        touch mod/forum/amd/src/local/layouts.js
        touch mod/forum/amd/src/local/layout/fullscreen.js
        touch mod/forum/styles.css
        

      6. Touch some more files and ensure that thy do not trigger a build:

        touch mod/assign/yui/src/history/js/history.js
        touch mod/assign/styles.css
        touch mod/assign/amd/src/grading_panel.js
        touch theme/boost/style/*.css
        touch theme/boost/scss/moodle/*.scss
        touch theme/boost/scss/moodle/editor.scss
        

        1. Confirm that there were no changes:

          git status
          

      Build a thirdpartylibs.xml file

      1. Run `grunt ignorefiles`
        1. Confirm that there were no changes:

          git status
          

      2. Open lib/thirdpartylibs.xml in your editor
      3. Go to the bottom of the file and add a new entry to a new directory, for example:

          <library>
            <location>foo</location>
            <name>foo</name>
            <license>GPL-3.0</license>
            <version>1.00</version>
          </library>
        

      4. Run `grunt ignorefiles`
        1. Confirm that the .stylelintignore and .eslintignore files were changed:

          git status
          

        2. Confirm that the new foo directory was added to both
      Show
      Build all from root directory Change directory into your moodle root Run grunt : grunt Confirm there were no files changes: git status Watch from root directory Change directory into your moodle root Run grunt watch : grunt watch Open a second terminal Touch some files. For each file confirm that the file builds. touch mod/forum/amd/src/discussion.js touch mod/forum/amd/src/local/layouts.js touch mod/forum/amd/src/local/layout/fullscreen.js touch mod/assign/yui/src/history/js/history.js touch mod/forum/styles.css touch theme/boost/scss/moodle/*.scss touch theme/boost/scss/moodle/editor.scss Touch some more files and ensure that thy do not trigger a build: touch theme/boost/style/*.css Confirm that there were no changes : git status Build all from component directory Change directory into mod/assign: cd mod/assign Run grunt : grunt Confirm there were no files changes: git status Watch from component directory Change directory into mod/assign: cd mod/assign Run grunt watch : grunt watch Open a second terminal Touch some files. For each file confirm that the file builds. touch mod/assign/yui/src/history/js/history.js touch mod/assign/styles.css touch mod/assign/amd/src/grading_panel.js Touch some more files and ensure that thy do not trigger a build: touch mod/forum/amd/src/discussion.js touch mod/forum/amd/src/local/layouts.js touch mod/forum/amd/src/local/layout/fullscreen.js touch theme/boost/style/*.css touch theme/boost/scss/moodle/*.scss touch theme/boost/scss/moodle/editor.scss touch mod/forum/styles.css Confirm that there were no changes : git status Watch from a different component directory Change directory into mod/forum: cd mod/forum Run grunt watch : grunt watch Open a second terminal Touch some files. For each file confirm that the file builds. touch mod/forum/amd/src/discussion.js touch mod/forum/amd/src/local/layouts.js touch mod/forum/amd/src/local/layout/fullscreen.js touch mod/forum/styles.css Touch some more files and ensure that thy do not trigger a build: touch mod/assign/yui/src/history/js/history.js touch mod/assign/styles.css touch mod/assign/amd/src/grading_panel.js touch theme/boost/style/*.css touch theme/boost/scss/moodle/*.scss touch theme/boost/scss/moodle/editor.scss Confirm that there were no changes : git status Watch and build from a theme directory Change directory into theme/boost cd theme/boost Run grunt : grunt Confirm there were no files changes: git status Run grunt watch Open a second terminal Touch some files. For each file confirm that the file builds. touch theme/boost/scss/moodle/*.scss touch theme/boost/scss/moodle/editor.scss Touch some more files and ensure that thy do not trigger a build: touch theme/boost/style/*.css Confirm that there were no changes : git status Watch and build from an AMD directory Change directory into mod/forum/amd: cd mod/forum/amd Run grunt : grunt Confirm that only the startup, eslint:amd, and babel:dist tasks ran Run grunt watch Open a second terminal Touch some files. For each file confirm that the file builds. touch mod/forum/amd/src/discussion.js touch mod/forum/amd/src/local/layouts.js touch mod/forum/amd/src/local/layout/fullscreen.js touch mod/forum/styles.css Touch some more files and ensure that thy do not trigger a build: touch mod/assign/yui/src/history/js/history.js touch mod/assign/styles.css touch mod/assign/amd/src/grading_panel.js touch theme/boost/style/*.css touch theme/boost/scss/moodle/*.scss touch theme/boost/scss/moodle/editor.scss Confirm that there were no changes : git status Build a thirdpartylibs.xml file Run `grunt ignorefiles` Confirm that there were no changes : git status Open lib/thirdpartylibs.xml in your editor Go to the bottom of the file and add a new entry to a new directory, for example: <library> <location>foo</location> <name>foo</name> <license>GPL-3.0</license> <version>1.00</version> </library> Run `grunt ignorefiles` Confirm that the .stylelintignore and .eslintignore files were changed : git status Confirm that the new foo directory was added to both

    Description

      When we run `grunt amd` it is very slow. It should not be. There's a bug:
      Every time we call the eslint or dist commands we are generating a list of files to work on. That list uses globstar (*) patterns like /amd/src//.j.

      Ultimately these are slow and non-performant. We already have a list of all subsystems, and plugintypes, and we can easily turn that into a list of components (subsystems + plugins + subplugins). From there we can create a smaller list whcih, while still using the globstar format, has a much smaller surface area.

      Whilst I discovered this with babel:dist, I also discovered that it applies to several other places, and there was some confusing behaviours too:

      • eslint:amd suffered the same/similar issue
      • eslint:yui suffered the same/similar issue
      • stylelint:css suffered the same/similar issue
      • stylelint:scss suffered the same/similar issue
      • in a directory [path/to/component] the grunt command had different behaviours:
        • grunt builds the whole of Moodle
        • grunt watch watches just the current directory
      • in a directory [path/to/component/amd] the grunt command had different behaviours:
        • grunt builds only the current directory
        • grunt watch claims to watch just the current directory but does nothing

      Therefore I've taken the liberty of fixing these issues too. They all had a similar root cause and similar fix.

      Attachments

        Issue Links

          Activity

            People

              dobedobedoh Andrew Lyons
              dobedobedoh Andrew Lyons
              Peter Dias Peter Dias
              Eloy Lafuente (stronk7) Eloy Lafuente (stronk7)
              Eloy Lafuente (stronk7) Eloy Lafuente (stronk7)
              Andrew Lyons, Huong Nguyen, Jun Pataleta, Michael Hawkins, Shamim Rezaie, Simey Lameze
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:
                15/Jun/20

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 7 hours, 27 minutes
                  7h 27m