Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-76174

Performance impact on course_overview block and my/courses.php

XMLWordPrintable

    • MOODLE_400_STABLE, MOODLE_401_STABLE
    • MOODLE_401_STABLE, MOODLE_402_STABLE
    • MDL-76174_401
    • Hide

      Initial setup

      1. Moodle instance with Xhprof installed and setup. Refer: https://docs.moodle.org/dev/Setting_up_xhprof_on_Moodle
      2. Turn Performance debug to on. Refer: https://docs.moodle.org/401/en/Debugging#Performance_info
      3. Copy maketestcategories.php to admin/tool/generator/cli/maketestcategories.php
      4. Create a student account called s1.
      5. Run CLI: php admin/tool/generator/cli/maketestcategories.php -t=1000 to create 1000 course categories.
      6. Login as admin. (This will be called Admin window)
      7. Navigate to Site Administration > Development > Profiling.
      8. Check to Enable profiling.
      9. Set the Profile these field to: /my/courses.php
      10. Click Save changes.
      11. Open Incognito mode on your browser.
      12. Login as student account s1 (This will be called Student window).

      Without patch

      1. Switch to Student window.
      2. Click on Dashboard.
      3. Run CLI: php admin/cli/purge_caches.php
      4. Click on My courses.
      5. Switch to Admin window.
      6. Navigate to Site Adminstration > Development > Profiling runs.
      7. Verify that you will see a record for /my/courses.php.
      8. Click on the Date column hyperlink.
      9. Note the Run ID for the record.
      10. Set the Mark as reference run/comment to: Without patch
      11. Click Save changes.

      With patch

      1. Pull the patch.
      2. Upgrade the Moodle instance.
      3. Switch to Student window.
      4. Click on Dashboard.
      5. Run CLI: php admin/cli/purge_caches.php
      6. Click on My courses.
      7. Reload the page.
      8. Switch to Admin window.
      9. Navigate to Site Adminstration > Developement > Profiling runs.
      10. Verify that you will see 2 new records for /my/courses.php.
      11. Click on the Date column hyperlink of the latest records.
      12. Set the Mark as reference run/comment to: Patched.
      13. Note the Run ID for Patched record.

      Comparison

      1. Switch to Admin window.
      2. Open the URL: [Moodle_Instance_URL]/admin/tool/profiling/index.php?runid=[Without_Patch_Run_id]&runid2=[Patched_Run_id]
      3. Verify that the Execution time and Function call of Patched are smaller than Without patch (Like the screenshot)

      Cleaning

      1. Delete the Test Parent Category to delete the 1000 created child categories.
      2. Turn off the Profiling.
      Show
      Initial setup Moodle instance with Xhprof installed and setup. Refer: https://docs.moodle.org/dev/Setting_up_xhprof_on_Moodle Turn Performance debug to on. Refer: https://docs.moodle.org/401/en/Debugging#Performance_info Copy maketestcategories.php to admin/tool/generator/cli/maketestcategories.php Create a student account called s1. Run CLI: php admin/tool/generator/cli/maketestcategories.php -t=1000 to create 1000 course categories. Login as admin. (This will be called Admin window) Navigate to Site Administration > Development > Profiling. Check to Enable profiling. Set the Profile these field to: /my/courses.php Click Save changes. Open Incognito mode on your browser. Login as student account s1 (This will be called Student window). Without patch Switch to Student window. Click on Dashboard. Run CLI: php admin/cli/purge_caches.php Click on My courses. Switch to Admin window. Navigate to Site Adminstration > Development > Profiling runs. Verify that you will see a record for /my/courses.php. Click on the Date column hyperlink. Note the Run ID for the record. Set the Mark as reference run/comment to: Without patch Click Save changes. With patch Pull the patch. Upgrade the Moodle instance. Switch to Student window. Click on Dashboard. Run CLI: php admin/cli/purge_caches.php Click on My courses. Reload the page. Switch to Admin window. Navigate to Site Adminstration > Developement > Profiling runs. Verify that you will see 2 new records for /my/courses.php. Click on the Date column hyperlink of the latest records. Set the Mark as reference run/comment to: Patched . Note the Run ID for Patched record. Comparison Switch to Admin window. Open the URL: [Moodle_Instance_URL] /admin/tool/profiling/index.php?runid= [Without_Patch_Run_id] &runid2= [Patched_Run_id] Verify that the Execution time and Function call of Patched are smaller than Without patch (Like the screenshot) Cleaning Delete the Test Parent Category to delete the 1000 created child categories. Turn off the Profiling.
    • 2
    • Team Hedgehog Sprint 1.2, Team Hedgehog 2023 Sprint 2.2

      In MDL-73549 a button was introduced so that users with the right to create new courses got a button to do so. 

      A function core_course_category::get_nearest_editable_subcategory() was introduced but this function has severe impact on performance if you have many course categories. On our 10.000+ course categories site a student would see 20 seconds load times.  (Running on 5 Nginx-servers and Redis-cachgin)(. 

      The function apperently traverser the entire course category structure in a very inefficient way where it continues to loop the categories until a category with "create course" capabilities is found. 

      This is especially critical since the majority of users (students) will not have any course creator rights and hence Moodle has to look though all course categories. 

      Steps to reproduce: 

      1. Turn on profiling 
      2. Create a large number of categories (10.000 +) in diffent levels.
      3. Log as a user without any rights to create courses (normally a student) 
      4. Oberserve the call grahp 

       

       

      Below is a profiling on a site with 2200 course categories. I have also attached the call graph. 

       

       

        1. callgraph 2022.svg
          97 kB
        2. comparison.png
          comparison.png
          186 kB
        3. explain_slow_mycourses.png
          explain_slow_mycourses.png
          204 kB
        4. image-2022-11-03-11-23-54-767.png
          image-2022-11-03-11-23-54-767.png
          19 kB
        5. image-2022-11-03-11-24-26-038.png
          image-2022-11-03-11-24-26-038.png
          16 kB
        6. maketestcategories.php
          2 kB
        7. MDL-76174_core_course__get_nearest_editable_subcategory_query_performance_improvement.patch
          2 kB
        8. MDL-76174.png
          MDL-76174.png
          392 kB
        9. Screenshot 2023-07-05 at 16.03.11.png
          Screenshot 2023-07-05 at 16.03.11.png
          209 kB

            bonczek Lars Bonczek
            tuekorsgaard Tue Korsgaard
            Stevani Andolo Stevani Andolo
            Ilya Tregubov Ilya Tregubov
            Ron Carl Alfon Yu Ron Carl Alfon Yu
            Votes:
            15 Vote for this issue
            Watchers:
            31 Start watching this issue

              Created:
              Updated:
              Resolved:

                Estimated:
                Original Estimate - 0 minutes
                0m
                Remaining:
                Remaining Estimate - 0 minutes
                0m
                Logged:
                Time Spent - 4 hours, 9 minutes
                4h 9m

                  Error rendering 'clockify-timesheets-time-tracking-reports:timer-sidebar'. Please contact your Jira administrators.