Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-66498

Analytics cleanup() causes error for large moodle installations under postgres

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 3.7.1, 3.8
    • 3.6.7, 3.7.3
    • Analytics
    • PostgreSQL, Microsoft SQL, Oracle
    • MOODLE_37_STABLE, MOODLE_38_STABLE
    • MOODLE_36_STABLE, MOODLE_37_STABLE
    • MDL-66498_master
    • Hide

      Manual testing instructions for 37 and master. The changes are unit tested.

      1. Go to Site Administration > Users > Accounts > Upload Users
      2. Upload the provided file to create 70.000 users.
      3. Start the cron (This will create the predictions for the "Upcoming activities due" model
      4. The task core\task\analytics_cleanup_task should not fail
      Show
      Manual testing instructions for 37 and master. The changes are unit tested. Go to Site Administration > Users > Accounts > Upload Users Upload the provided file to create 70.000 users. Start the cron (This will create the predictions for the "Upcoming activities due" model The task core\task\analytics_cleanup_task should not fail

    Description

      The analytics API regularly cleans up the prediction and training samples. For that, it creates an sql statement, which has a NOT IN within its where clause. For this NOT IN it includes all ids of the analyzable of the model as parameters. For our installation and the analyzable core_user this are nearly 100,000 entries. However, the postgres implementation in php only allows 65536 parameters. Other DBMS have similar issues. I could not find a limit for mysql, but SQL Server is at 32767 (source ). Thus, the query fails

      Attachments

        Issue Links

          has a non-specific relationship to

          Improvement - An enhancement to an existing Moodle feature. MDL-53735 Split very large SQL-IN parameters to chunks

          • Minor - Minor loss of function where workaround is possible
          • Closed
          has been marked as being related by

          Bug - A problem which impairs or prevents Moodle from functioning correctly. MDL-67440 \core\task\analytics_cleanup_task extremely slow on postgres site.

          • Minor - Minor loss of function where workaround is possible
          • Closed

          Activity

            People

              dmonllao David Monllaó
              tobias.reischmann Tobias Reischmann
              Tobias Reischmann Tobias Reischmann
              Andrew Lyons Andrew Lyons
              Janelle Barcega Janelle Barcega
              Votes:
              5 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:
                11/Nov/19

                Time Tracking

                  Estimated:
                  Original Estimate - 0 minutes
                  0m
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 2 days, 2 hours, 25 minutes
                  2d 2h 25m