Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-57852

Disable cron at site config level via gui and cli with fast shutdown

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Development in progress
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 3.3
    • Fix Version/s: None
    • Component/s: Tasks
    • Testing Instructions:
      Hide

      Automated Testing:

      • PHPUnit
      • Bash tests: install https://github.com/catalyst/moodle-tool_testtasks and run all tests inside the tests folder. Descriptions are as follows:
        test1.sh: Assert scheduled tasks won't run if disabled. Assert admin/tool/task/cli/schedule_task.php will still run.
        test2.sh: Assert cron will finish early if disabled while running.
        test3.sh: Assert specifically that adhoc tasks will finish early.
        test4.sh: Assert is-running should return is running in under < 1 sec
        test5.sh: Assert is-running verbose should provide details on task lock that could not be acquired.
        test6.sh: disable-wait will behave properly with two parallel crons running. This script has no assertions built in - watch the output and make sure it behaves properly - one lock for each cron should be unavailable and it should keep retying the locks till they are acquired and then display that cron is not running.

      Manual Testing:

      • Run through each of the new cli commands
      • ensure the GUI enable/disable works on the scheduled_tasks page
      Show
      Automated Testing: PHPUnit Bash tests: install https://github.com/catalyst/moodle-tool_testtasks and run all tests inside the tests folder. Descriptions are as follows: test1.sh: Assert scheduled tasks won't run if disabled. Assert admin/tool/task/cli/schedule_task.php will still run. test2.sh: Assert cron will finish early if disabled while running. test3.sh: Assert specifically that adhoc tasks will finish early. test4.sh: Assert is-running should return is running in under < 1 sec test5.sh: Assert is-running verbose should provide details on task lock that could not be acquired. test6.sh: disable-wait will behave properly with two parallel crons running. This script has no assertions built in - watch the output and make sure it behaves properly - one lock for each cron should be unavailable and it should keep retying the locks till they are acquired and then display that cron is not running. Manual Testing: Run through each of the new cli commands ensure the GUI enable/disable works on the scheduled_tasks page
    • Affected Branches:
      MOODLE_33_STABLE
    • Pull Master Branch:
      MDL-57852-master

      Description

      In the 2.7+ schedule task api we can disable a task from running, but we cannot stop the whole cron system from running. You can of course disable this at the crontab or cron system level but depending on how your env's are setup this can be awkward. eg a bunch of dockers / auto scaled front ends and cron is running concurrently everywhere. You don't want to have to do this at the infrastructure level, it should be an application level concern.

      There are a couple reasons why you'd want this, but the main one driving it is continuous deployments. We want them to be as quick as possible, and minimize the actual downtime. So we want to disable cron and let the current running processes finish and only then do a deployment and upgrade. But we do NOT want to enable maintenance mode until the last minute to minimize the downtime. There are other use cases like just turning off cron when diagnosing a production issue.

      So I'd just want a very simple flag which can be set and unset from a cli. It would be good to expose it in the gui on the schedule task page as well. A very important feature is that the task system would also check for this flag between every task, so when it is set it kills cron as soon as possible. Some behemoth potentially long running tasks could optionally also periodically check this flag and decide to exit early as well. This is all in the interests of minimizing the total overall time to deploy.

      Ideally aditional cli which has these options:

      admin/cli/cron.php -h

      cron.php --isrunning    # test if cron is running on anywhere on any box
      cron.php --disable      # disables cron, tells other running crons to stop early
      cron.php --enable       # enables cron again
      cron.php --disable-wait # disables cron, and waits until all other crons have finished
      cron.php --verbose      # Print verbose task information when coupled with disable-wait and is-running
      

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                6 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated: