Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-61519

Calendar should not iterate through all categories

    XMLWordPrintable

    Details

    • Testing Instructions:
      Hide
      Before the patch
      1. Checkout the last weekly release before the patch has been integrated.
        • For master:

          git checkout b63a3b04b154990497bd58255b4ed579d37b432a
          

        • For 3.4:

          git checkout b63a3b04b154990497bd58255b4ed579d37b432a
          

      2. Enable performance information in the footer.
        Note: If using MDK, run

        mdk run mindev

      3. Create three course categories, and one course in each category
      4. Add the blocks Upcoming Events and Calendar to the dashboard if not already there.
      5. View the dashboard and take note of the number of DB reads and the writes to the core/coursecatrecords cache.
      Testing with the patch
      1. Checkout the latest integration branch.
        • For master:

          git checkout master

        • For 3.4:

          git checkout MOODLE_34_STABLE

      2. Purge caches
      3. View the dashboard with the patch and notice that both numbers decreased.
      4. View the dashboard again and see that the course categories for the calendar are now fetched from the core/calendar_categories cache.
      5. Create a course category event for each of the categories
      6. Create a new user and login as that user in a different browser
        1. Confirm that you don't see any of the course category events
      7. Enrol that user in the course in the first category
      8. As the second user, refresh the calendar
        1. Confirm that you now see the events for the first category
        2. Confirm that you don't see any of the course category events for the second or third categories
      9. As admin, open Site admin -> courses -> Manage courses and categories
      10. Find the second category, click on its cog icon, and choose "Assign roles"
      11. Assign the Manager role to your other user
      12. As the second user, refresh the calendar
        1. Confirm that you see the events for the first category
        2. Confirm that you see the events for the second category
        3. Confirm that you DO NOT see the events for the third category
      Show
      Before the patch Checkout the last weekly release before the patch has been integrated. For master: git checkout b63a3b04b154990497bd58255b4ed579d37b432a For 3.4: git checkout b63a3b04b154990497bd58255b4ed579d37b432a Enable performance information in the footer. Note: If using MDK, run mdk run mindev Create three course categories, and one course in each category Add the blocks Upcoming Events and Calendar to the dashboard if not already there. View the dashboard and take note of the number of DB reads and the writes to the core/coursecatrecords cache. Testing with the patch Checkout the latest integration branch. For master: git checkout master For 3.4: git checkout MOODLE_34_STABLE Purge caches View the dashboard with the patch and notice that both numbers decreased. View the dashboard again and see that the course categories for the calendar are now fetched from the core/calendar_categories cache. Create a course category event for each of the categories Create a new user and login as that user in a different browser Confirm that you don't see any of the course category events Enrol that user in the course in the first category As the second user, refresh the calendar Confirm that you now see the events for the first category Confirm that you don't see any of the course category events for the second or third categories As admin, open Site admin -> courses -> Manage courses and categories Find the second category, click on its cog icon, and choose "Assign roles" Assign the Manager role to your other user As the second user, refresh the calendar Confirm that you see the events for the first category Confirm that you see the events for the second category Confirm that you DO NOT see the events for the third category
    • Affected Branches:
      MOODLE_34_STABLE
    • Fixed Branches:
      MOODLE_34_STABLE
    • Pull from Repository:
    • Pull Master Branch:
      MDL-61519-master

      Description

      The calendar currently has several calls to \coursecat::get_all() which is especially inefficient when you have a lot of course categories (more than CONTEXT_CACHE_MAX_SIZE that is 2500 currently).

      calendar_information->set_sources and core_calendar_external->get_calendar_events even have nested iterations through all categories, which doesn't scale well.

      We have about 3000 course categories and loading the dashboard takes about 8 seconds and more than 10000 db queries are executed.

      These are the backtraces for the queries:

      * line 776 of /lib/dml/pgsql_native_moodle_database.php: call to pgsql_native_moodle_database->query_end()
      * line 1571 of /lib/dml/moodle_database.php: call to pgsql_native_moodle_database->get_records_sql()
      * line 1543 of /lib/dml/moodle_database.php: call to moodle_database->get_record_sql()
      * line 1522 of /lib/dml/moodle_database.php: call to moodle_database->get_record_select()
      * line 6192 of /lib/accesslib.php: call to moodle_database->get_record()
      * line 2462 of /lib/coursecatlib.php: call to context_coursecat::instance()
      * line 2411 of /lib/coursecatlib.php: call to coursecat->get_context()
      * line 72 of /cache/lib.php: call to coursecat->prepare_to_cache()
      * line 711 of /cache/classes/loaders.php: call to cache_cached_object->__construct()
      * line 330 of /lib/coursecatlib.php: call to cache->set_many()
      * line 1181 of /calendar/lib.php: call to coursecat::get_all()
      * line 1079 of /calendar/lib.php: call to calendar_information->set_sources()
      * line 52 of /blocks/calendar_upcoming/block_calendar_upcoming.php: call to calendar_information::create()
      

      * line 776 of /lib/dml/pgsql_native_moodle_database.php: call to pgsql_native_moodle_database->query_end()
      * line 1571 of /lib/dml/moodle_database.php: call to pgsql_native_moodle_database->get_records_sql()
      * line 1543 of /lib/dml/moodle_database.php: call to moodle_database->get_record_sql()
      * line 1522 of /lib/dml/moodle_database.php: call to moodle_database->get_record_select()
      * line 6192 of /lib/accesslib.php: call to moodle_database->get_record()
      * line 2462 of /lib/coursecatlib.php: call to context_coursecat::instance()
      * line 1182 of /calendar/lib.php: call to coursecat->get_context()
      * line 1079 of /calendar/lib.php: call to calendar_information->set_sources()
      * line 52 of /blocks/calendar_upcoming/block_calendar_upcoming.php: call to calendar_information::create()
      

      When you have so many categories, iterating through all categories constantly fills up the context cache. When you want to access a category context, the cache entry has already been discarded.

        Attachments

        1. Before the patch_Step5.PNG
          74 kB
          Gladys Basiana
        2. caches.png
          98 kB
          Jan Dageförde
        3. Testing with the patch_Step12_No.1,2,3 Confirmed.PNG
          108 kB
          Gladys Basiana
        4. Testing with the patch_Step3.PNG
          78 kB
          Gladys Basiana
        5. Testing with the patch_Step4.PNG
          75 kB
          Gladys Basiana
        6. Testing with the patch_Step6.PNG
          77 kB
          Gladys Basiana
        7. Testing with the patch_Step8_No.1 & 2 Confirmed.PNG
          123 kB
          Gladys Basiana

          Activity

            People

            Assignee:
            mgauk Martin Gauk
            Reporter:
            mgauk Martin Gauk
            Peer reviewer:
            Jan Dageförde
            Integrator:
            Andrew Nicols
            Tester:
            Gladys Basiana
            Participants:
            Component watchers:
            Andrew Nicols, Jun Pataleta, Michael Hawkins, Shamim Rezaie, Simey Lameze, Matteo Scaramuccia, Andrew Nicols, Jun Pataleta, Michael Hawkins, Shamim Rezaie, Simey Lameze
            Votes:
            9 Vote for this issue
            Watchers:
            7 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:
              Fix Release Date:
              17/May/18

                Time Tracking

                Estimated:
                Original Estimate - 0 minutes
                0m
                Remaining:
                Remaining Estimate - 0 minutes
                0m
                Logged:
                Time Spent - 3 hours
                3h