Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-66498

Analytics cleanup() causes error for large moodle installations under postgres

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 3.7.1, 3.8
    • 3.6.7, 3.7.3
    • Analytics
    • PostgreSQL, Microsoft SQL, Oracle
    • MOODLE_37_STABLE, MOODLE_38_STABLE
    • MOODLE_36_STABLE, MOODLE_37_STABLE
    • MDL-66498_master
    • Hide

      Manual testing instructions for 37 and master. The changes are unit tested.

      1. Go to Site Administration > Users > Accounts > Upload Users
      2. Upload the provided file to create 70.000 users.
      3. Start the cron (This will create the predictions for the "Upcoming activities due" model
      4. The task core\task\analytics_cleanup_task should not fail
      Show
      Manual testing instructions for 37 and master. The changes are unit tested. Go to Site Administration > Users > Accounts > Upload Users Upload the provided file to create 70.000 users. Start the cron (This will create the predictions for the "Upcoming activities due" model The task core\task\analytics_cleanup_task should not fail

    Description

      The analytics API regularly cleans up the prediction and training samples. For that, it creates an sql statement, which has a NOT IN within its where clause. For this NOT IN it includes all ids of the analyzable of the model as parameters. For our installation and the analyzable core_user this are nearly 100,000 entries. However, the postgres implementation in php only allows 65536 parameters. Other DBMS have similar issues. I could not find a limit for mysql, but SQL Server is at 32767 (source ). Thus, the query fails

      Attachments

        Issue Links

          has a non-specific relationship to

          Improvement - An enhancement to an existing Moodle feature. MDL-53735 Split very large SQL-IN parameters to chunks

          • Minor - Minor loss of function where workaround is possible
          • Closed
          has been marked as being related by

          Bug - A problem which impairs or prevents Moodle from functioning correctly. MDL-67440 \core\task\analytics_cleanup_task extremely slow on postgres site.

          • Minor - Minor loss of function where workaround is possible
          • Closed

          Activity

            People

              dmonllao David Monllaó
              tobias.reischmann Tobias Reischmann
              Tobias Reischmann Tobias Reischmann
              Andrew Lyons Andrew Lyons
              Janelle Barcega Janelle Barcega
              David Woloszyn, Huong Nguyen, Jake Dallimore, Meirza, Michael Hawkins, Raquel Ortega, Safat Shahin, Stevani Andolo
              Votes:
              5 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:
                11/Nov/19

                Time Tracking

                  Estimated:
                  Original Estimate - 0 minutes
                  0m
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 2 days, 2 hours, 25 minutes
                  2d 2h 25m