Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-67597

Blocking cron tasks only effectively lock into the future not the past, ie race condition

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 3.5.9, 3.6.7, 3.7.3, 3.8
    • Fix Version/s: None
    • Component/s: Tasks
    • Affected Branches:
      MOODLE_35_STABLE, MOODLE_36_STABLE, MOODLE_37_STABLE, MOODLE_38_STABLE

      Description

      Lets say you have a normal cron task A which take 1 minutes to run, and a blocking task B which also takes 1 minute to run. I believe the design intent of the is_blocking() flag is so that task B will never be run concurrently with any other task, and it enforces this by holding onto the global cron lock so no other task will start until B finishes. But it doesn't do any locking or detection for tasks which are already running. So if A starts and then B then they will run side by side. If B starts first then A is blocked.

      A proposed solution split from MDL-67486:

      If there are no blocking tasks in either queue, then we can just not do any global cron lock gaining at all. Then as soon as we see any blocking task in either queue, then we effectively shut down cron almost completely and just wait until no other tasks are running, so most processes will just exit. Then after a few minutes when no other tasks are running then we can run all the blocking tasks in serial (usually only a single one), and then commence normal operations. At the moment this isn't possible because we have no way of querying what is currently running, but after MDL-67211 lands we will have this metadata to be able to safely do this check.

       

      Eg if you run this in two terminals then A will block B:

      A) php admin/tool/task/cli/schedule_task.php --execute='\core\task\create_contexts_task'

      B) php admin/tool/task/cli/schedule_task.php --execute='\tool_testtasks\task\slow_task'

       

      If you run them in the other order then A should block B but does not:

      A) php admin/tool/task/cli/schedule_task.php --execute='\tool_testtasks\task\slow_task'

      B) php admin/tool/task/cli/schedule_task.php --execute='\core\task\create_contexts_task'

       

      In practice in any large instance where there are multiple cron processes running, the chance of create_contexts_task being started when no other task is running approaches zero as you add scale.

       

       

       

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              brendanheywood Brendan Heywood
              Participants:
              Component watchers:
              Amaia Anabitarte, Carlos Escobedo, Ferran Recio, Sara Arjona (@sarjona)
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated: