Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-82584

Workshop get_users_with_capability_sql method optimization

XMLWordPrintable

    • Icon: Improvement Improvement
    • Resolution: Fixed
    • Icon: Minor Minor
    • 5.0
    • 4.3.5, 4.4, 4.5
    • Workshop

      The workshop activity module has a performance issue related to its method, get_users_with_capability_sql, which generates a poorly optimized SQL query. Currently, it uses a UNION to combine SQL queries, but this could be improved by implementing simple caching and avoiding complex UNION queries.

      For instance, when there are 1000 groups, loading content becomes significantly slow and may even result in a 504 error due to timeout. I have attached a text file containing a SQL query which spans 15,000 lines, along with an optimized version of the SQL. Initially, I encountered a different problem that I couldn't replicate locally, but this example is the closest scenario I've managed to recreate the issue.

      Edit (Conn Warwicker)

      I have taken this ticket from Vlad now as he had to work on other things.

      Here's what I've done:

      • Removed the calls to some of the core functions to build up the SQL, as this was creating huge SQL statements, e.g. 80,000+ lines long in some cases, with endless UNIONS and other stuff which simply wasn't needed.
      • Tried to streamline it all into 1 simple SQL query custom to the workshhop module
      • I did initially play around with doing the bare minimum in SQL and then running users through `has_capability()` but it causes memory errors on very large courses.

       

      Bench-marks:

      (Purged cache before each test)

      Pre-patch:

      • Workshop view page: (25s)
      • Workshop submissions allocation page [10 users per page]: (45s)

      With-patch:

      • Workshop view page: (9s)
      • Workshop submissions allocation page [10 users per page]: (16s)

       

      Screenshots:

      pre-patch-view-page

      pre-patch-allocation-page

      with-patch-view-page

      with-patch-allocation-page

       

       

        1. (1) 6 Passed -- (Main)MDL-82584.png
          (1) 6 Passed -- (Main)MDL-82584.png
          344 kB
        2. pre-patch-allocation-page
          111 kB
        3. pre-patch-view-page
          116 kB
        4. sql_after_improvement.txt
          113 kB
        5. sql_before_improvement.txt
          1.14 MB
        6. with-patch-allocation-page
          132 kB
        7. with-patch-view-page
          189 kB

            cwarwicker C Warwicker
            vladaskidanovas vladaskidanovas
            Mark Johnson Mark Johnson
            Amaia Anabitarte Amaia Anabitarte
            Kim Jared Lucas Kim Jared Lucas
            Votes:
            1 Vote for this issue
            Watchers:
            9 Start watching this issue

              Created:
              Updated:
              Resolved:

                Estimated:
                Original Estimate - 0 minutes
                0m
                Remaining:
                Remaining Estimate - 0 minutes
                0m
                Logged:
                Time Spent - 1 day, 50 minutes
                1d 50m

                  Error rendering 'clockify-timesheets-time-tracking-reports:timer-sidebar'. Please contact your Jira administrators.