Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-68520

Improve task manager - adhoc task on medium/large scale

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Minor
    • Resolution: Duplicate
    • Affects Version/s: 3.7.5
    • Fix Version/s: None
    • Component/s: Performance, Tasks
    • Labels:
      None
    • Affected Branches:
      MOODLE_37_STABLE

      Description

      We are working with moodle release 3.7.3+ (Build: 20200103) , we can't update now because 'office private reasons', but I notice that this can help other people in the same situation and the last stable 3.7.5 version does not include the changes that I indicate below.

      We have cron tasks running each minute, but for a reason I don´t know the table mdl_task_adhoc get bigger until 650 thousands rows. Then, mails (forum notifications, etc) started to send days after, even a week and the table size continued to increase in size because the daily activity in campus continued. Events added more faster than mails can dispatch.

      On the other side task file locking time get bigger and runners overlapping each other.

      Not SMTP issue, the server only receives one mail every 3 seconds from then Moodle server, then I started to debug task manager code and I notice that for each get_next_adhoc_task in lib/classes/task/manager.php the query get all the records from the table, select the next task and return to the manager, getting 650 thousands records each time, for each next task processing, getting slower all the proccess.

      What I did is setting a limit to the query adding last two parameters to get_record_select and get only 500 records, then the sending of emails was normalized after hours, because after this change our SMTP server step from receiving 20 emails per minute to 250/300 (or more).

      Affected line in /lib/classes/task/manager.php function get_next_adhoc_task, line 493:

          # vim <moodledir>/lib/classes/task/manager.php +493
          //$records = $DB->get_records_select('task_adhoc', $where, $params);
          $records = $DB->get_records_select('task_adhoc', $where, $params, '', '*', 0, 500);
      

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              Unassigned
              Reporter:
              pmaseda Pablo Maseda
              Participants:
              Component watchers:
              Matteo Scaramuccia, Andrew Nicols, Jun Pataleta, Michael Hawkins, Shamim Rezaie, Simey Lameze, Amaia Anabitarte, Carlos Escobedo, Ferran Recio, Sara Arjona (@sarjona), Víctor Déniz Falcón
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: