Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-83718

Improve reportbuilder table performance by combining count + select

XMLWordPrintable

      1. In Moodle 4.4 access report builder page /reportbuilder/index.php
      2. Click on "New Report" and name it "Enrolment Count".
      3. Set the "Report Source" to "Course Participants".
      4. Configure the columns as follows:

           - Course • Course ID number - No aggregation
           - Course • Course short name - No aggregation
           - Course • Course full name - No aggregation
           - Enrolment method • Name - No aggregation
           - Tenant • Tenant name - No aggregation
           - User • Username - No aggregation
           - Enrolment method • Plugin - No aggregation
           - Course • Course short name with link - No aggregation
           - Course • Course full name with link - No aggregation
        

      5. It takes ~10 seconds to save the changes and render data
      6. After setting "User • Username" to "Count distinct" it takes more than 2 minutes to save the changed and render data. Same 2+ minutes are required to view the report
      7. Setting "User • Username" to "Count" increases execution time from 10 seconds up to 20 which is kinda workable

      Here is the tool_excimer flame graph:

      We could debug and catch the actual SQL that is being executed, see MDL-83718-original-sql.txt attached. Running it directly in the database gives the same timing: ~60 seconds (2 mins devided by 2 as RB basically runs the SQL twice - to get the count and then get the actual records).

      The simplified version of the SQL takes 40 seconds to complete:

      SELECT
          rbalias3.idnumber AS c0_idnumber,
          rbalias3.shortname AS c1_shortname,
          rbalias3.id AS c1_courseid,
          rbalias3.fullname AS c2_fullname,
          rbalias3.id AS c2_courseid,
          rbalias8.enrol AS c3_enrol,
          rbalias8.name AS c3_name,
          rbalias8.courseid AS c3_courseid,
          rbalias8.roleid AS c3_roleid,
          rbalias8.customint1 AS c3_customint1,
          rbalias8.enrol AS c5_enrol,
          COUNT(DISTINCT rbalias15.username) AS c6_username,
          rbalias3.shortname AS c7_courseshortnamewithlink,
          rbalias3.id AS c7_id,
          rbalias3.fullname AS c8_coursefullnamewithlink,
          rbalias3.id AS c8_id
      FROM
          mdl_course rbalias3
      LEFT JOIN
          mdl_context rbalias4 ON rbalias4.contextlevel = 50 AND rbalias4.instanceid = rbalias3.id
      LEFT JOIN
          mdl_enrol rbalias8 ON rbalias8.courseid = rbalias3.id
      LEFT JOIN
          mdl_user_enrolments rbalias9 ON rbalias9.userid <> 1
          AND rbalias9.enrolid = rbalias8.id
      LEFT JOIN
          mdl_user rbalias15 ON rbalias15.id = rbalias9.userid AND rbalias15.deleted = 0
      WHERE
          rbalias3.id != 1
      GROUP BY
          c0_idnumber, 
          c3_enrol, c3_name, c3_courseid, c3_roleid, c3_customint1, c5_enrol,
          c7_courseshortnamewithlink, c7_id
      ORDER BY
          c3_enrol ASC;
      

      Without the COUNT DISTINCT the SQL completes within seconds.

      Moodle site details: Moodle 4.4.4, MySQL 8.0.36, PHP 8.1.2.1.2.19
      Number of records in the involved tables:

      mdl_course - 500
      mdl_context - 40K
      mdl_enrol - 4K
      mdl_user_enrolments - 550K
      mdl_user - 33K
      

            pholden Paul Holden
            mikhailgolenkov Misha Golenkov
            Carlos Castillo Carlos Castillo
            David Carrillo David Carrillo
            CiBoT CiBoT
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved:

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0 minutes
                0m
                Logged:
                Time Spent - 2 hours, 34 minutes
                2h 34m

                  Error rendering 'clockify-timesheets-time-tracking-reports:timer-sidebar'. Please contact your Jira administrators.