Cron / adhoc task runners ramp up slowly for no reason

1. Install testtasks to make testing easier:

   git clone https://github.com/catalyst/moodle-tool_testtasks admin/tool/testtasks

2. Install lockstats to make testing easier:

   git clone https://github.com/catalyst/moodle-tool_lockstats admin/tool/lockstats

3. Add this to config.php

   $CFG->lock_factory = '\tool_lockstats\proxy_lock_factory';

   $CFG->proxied_lock_factory = 'auto';

4. In theory allow cron to scale right up

   $CFG->task_adhoc_concurrency_limit = 1000;

   $CFG->task_scheduled_concurrency_limit = 1000;

   $CFG->task_adhoc_max_runtime = 6000;

5. In terminal 1 watch for open locks, should be empty:

   watch -n 1 php admin/tool/lockstats/cli/list_locks.php

6. In terminal 2 queue up a ton of adhoc tasks:

   php admin/tool/testtasks/cli/queue_adhoc_tasks.php -d=5 -n=900

7. In terminal 2 spawn a new adhoc runner process:

   php admin/tool/task/cli/adhoc_task.php --execute &

   1. Confirm that you see runner and adhoc tasks in terminal 1 within a few seconds
8. Terminal 2: Add a few more runners by running the following:

   php admin/tool/task/cli/adhoc_task.php --execute &

   php admin/tool/task/cli/adhoc_task.php --execute &

   php admin/tool/task/cli/adhoc_task.php --execute &

   1. Terminal 1: Confirm that they appear in the task list immediately
9. Kill all the background processes:

   php admin/cli/cron.php --stop

10. Now go crazy and ramp it right up

    for i in {1..30}; do php admin/tool/task/cli/adhoc_task.php --execute & done

    1. Confirm that you see runner and adhoc tasks in terminal 1 within a few seconds 

Note

There are some race conditions in the lockstats plugin. You may seem some PHP errors pop up in the output as it flies by your screen.