Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-50009

Prevent scheduled tasks from leaving unfinished db transactions

XMLWordPrintable

    • MOODLE_28_STABLE, MOODLE_29_STABLE
    • MOODLE_30_STABLE
    • Hide
      1. Install the attached test plugin and go to /admin/tool/mdl50009 to try to break your task queue
      2. Run your cron job
      3. Confirm that the task causes an error message to appear in the cron log
      4. Confirm that the record in task_adhoc hasn't been removed after the cron job finishes
      5. Raise some other task and repeat
      6. Confirm that the second task hasn't been prevented from running by the broken first task.
      Show
      Install the attached test plugin and go to /admin/tool/mdl50009 to try to break your task queue Run your cron job Confirm that the task causes an error message to appear in the cron log Confirm that the record in task_adhoc hasn't been removed after the cron job finishes Raise some other task and repeat Confirm that the second task hasn't been prevented from running by the broken first task.

      It is possible for a badly-coded task class to cause havoc with the scheduled and ad-hoc task system. Specifically, if a task starts a transaction and doesn't allow commit on it, anything done by that run of the cron job will be rolled back when the script finishes running, so:

      • no tasks will ever be completed and removed successfully from the queue
      • any tasks which do anything outside the database will do it every time the cron runs (forever)

      At worst, the cron script should probably check for started transactions before and after running a task, and log an error message and roll back if one is found. Ideally, each task should have its own transaction so it is unable to break the task processing.

      I realise you could take the hard line that systems administrators should ensure that all plugins containing tasks always complete transactions correctly in all possible code paths, etc, (which is mostly true of course!) but I don't think this is reasonable in this case, for a few reasons:

      • the cron log shows no sign of the error so it appears that it has run correctly (there is a message in the error_log, but not the cron log)
      • there is no user interface for monitoring the ad-hoc task queue, so you may only notice the problem if you were actively monitoring the database table
      • many institutions are required to use a particular third-party plugin, and may not be able to change the code without invalidating a support contract. In these cases, you expect the problems caused by buggy code to be limited to the plugin's own functionality rather than affecting the whole system.

      This last issue is one we have encountered with an integration with a commercial service in the past. There were various bugs in its event handling which resulted in the (2.6) events queue becoming blocked, which luckily only affected that particular plugin. This would have been a nightmare if the event queue had become blocked for all plugins, which is broadly similar to what is happening with the task queue at the moment.

      (Note - filing this against "other" component as there is currently no component for the task API)

            maherne Michael Aherne
            maherne Michael Aherne
            Simey Lameze Simey Lameze
            Dan Poltawski Dan Poltawski
            Rajesh Taneja Rajesh Taneja
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved:

                Error rendering 'clockify-timesheets-time-tracking-reports:timer-sidebar'. Please contact your Jira administrators.