Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-67363

Add a Quality of Service layer to the processing of the ad-hoc task queue

XMLWordPrintable

    • MOODLE_39_STABLE
    • MOODLE_39_STABLE
    • MDL-67363-adhoc-qos
    • Hide

      0) Install this in admin/tool to make the testing easier:

      https://github.com/catalyst/moodle-tool_testtasks

       

      Testing scenario 1. Test the case with a single task runner

      1) Queue up 1000 one seconds tasks:

      php admin/tool/testtasks/cli/queue_adhoc_tasks.php -d=1 -n=1000
      

      2) Behind this queue up a single 'another' type of adhoc task:

      php admin/tool/testtasks/cli/queue_adhoc_tasks.php -d=1 -n=1 --class='tool_testtasks\task\another_timed_adhoc_task'

      3) Peek into the task queue by class:

      $ select count(*),classname from mdl_task_adhoc group by classname;
       count | classname 
      -------+-----------------------------------------------
       1000 | \tool_testtasks\task\timed_adhoc_task
       1 | \tool_testtasks\task\another_timed_adhoc_task
      (2 rows)

      4) Now start processing the queue:

      php admin/tool/task/cli/adhoc_task.php --execute 
      

      5) You should see that the first 2 tasks it picked off cycled through each type and very quickly you are left with a number lower than 1000 (i.e. 997):

      $ select count(*),classname from mdl_task_adhoc group by classname;
       count | classname 
      -------+---------------------------------------
       997 | \tool_testtasks\task\timed_adhoc_task
      (1 row)

       

      Testing scenario 2. A bunch of runners in parallel

      1) First, allow this to work in config.php:

      $CFG->task_adhoc_concurrency_limit = 1000;
      

      2) Queue up 1000 of type A and then another 1000 of type B behind it:

      $ php admin/tool/testtasks/cli/clear_adhoc_task_queue.php
      $ php admin/tool/testtasks/cli/queue_adhoc_tasks.php -d=1 -n=1000
      $ php admin/tool/testtasks/cli/queue_adhoc_tasks.php -d=1 -n=1000 --class='tool_testtasks\task\another_timed_adhoc_task' 
      

      3) Confirm the queues: 

      $ select count(*),classname from mdl_task_adhoc group by classname;
       count | classname 
      -------+-----------------------------------------------
       1000 | \tool_testtasks\task\timed_adhoc_task
       1000 | \tool_testtasks\task\another_timed_adhoc_task
      (2 rows)
      

      6) Now lets fire up several task runners, do this say 4 times (you may need to open several terminals in order to execute them):

      php admin/tool/task/cli/adhoc_task.php --execute &
      

      7) Recheck the queues and confirm they are being processed evenly (so the count column number is lower than in step #3):

      $ select count(*),classname from mdl_task_adhoc group by classname;
       count | classname 
      -------+-----------------------------------------------
       944 | \tool_testtasks\task\timed_adhoc_task
       944 | \tool_testtasks\task\another_timed_adhoc_task
      (2 rows) 
      

       

      Testing scenario 3. More cases [OPTIONAL]

      8) Lastly, and most importantly, keep throwing more and more runners at it and confirm that the overall system is still scaling up linearly the more processes you throw at it.

       
       

      Show
      0) Install this in admin/tool to make the testing easier: https://github.com/catalyst/moodle-tool_testtasks   Testing scenario 1. Test the case with a single task runner 1) Queue up 1000 one seconds tasks: php admin /tool/testtasks/cli/queue_adhoc_tasks .php -d=1 -n=1000 2) Behind this queue up a single 'another' type of adhoc task: php admin /tool/testtasks/cli/queue_adhoc_tasks .php -d=1 -n=1 --class= 'tool_testtasks\task\another_timed_adhoc_task' 3) Peek into the task queue by class: $ select count(*),classname from mdl_task_adhoc group by classname; count | classname -------+----------------------------------------------- 1000 | \tool_testtasks\task\timed_adhoc_task 1 | \tool_testtasks\task\another_timed_adhoc_task (2 rows) 4) Now start processing the queue: php admin /tool/task/cli/adhoc_task .php --execute  5) You should see that the first 2 tasks it picked off cycled through each type and very quickly you are left with a number lower than 1000 (i.e. 997): $ select count(*),classname from mdl_task_adhoc group by classname; count | classname -------+--------------------------------------- 997 | \tool_testtasks\task\timed_adhoc_task (1 row)   Testing scenario 2. A bunch of runners in parallel 1) First, allow this to work in config.php: $CFG ->task_adhoc_concurrency_limit = 1000; 2) Queue up 1000 of type A and then another 1000 of type B behind it: $ php admin /tool/testtasks/cli/clear_adhoc_task_queue .php $ php admin /tool/testtasks/cli/queue_adhoc_tasks .php -d=1 -n=1000 $ php admin /tool/testtasks/cli/queue_adhoc_tasks .php -d=1 -n=1000 --class= 'tool_testtasks\task\another_timed_adhoc_task'   3) Confirm the queues:  $ select count(*),classname from mdl_task_adhoc group by classname; count | classname -------+----------------------------------------------- 1000 | \tool_testtasks\task\timed_adhoc_task 1000 | \tool_testtasks\task\another_timed_adhoc_task ( 2 rows) 6) Now lets fire up several task runners, do this say 4 times (you may need to open several terminals in order to execute them): php admin /tool/task/cli/adhoc_task .php --execute & 7) Recheck the queues and confirm they are being processed evenly (so the count column number is lower than in step #3): $ select count(*),classname from mdl_task_adhoc group by classname; count | classname -------+----------------------------------------------- 944 | \tool_testtasks\task\timed_adhoc_task 944 | \tool_testtasks\task\another_timed_adhoc_task ( 2 rows)    Testing scenario 3. More cases [OPTIONAL] 8) Lastly, and most importantly, keep throwing more and more runners at it and confirm that the overall system is still scaling up linearly the more processes you throw at it.    

      This is split from MDL-64610

      Suppose you have 10 ad hoc task runners and 1000 type A tasks in the queue, then we want to run 10 type A side by side.

      Suppose you have 10 ad hoc task runners and 1000 type B tasks in the queue, then we want to run 10 type B side by side.

      Suppose you have 10 ad hoc task runners and 500 A and then 500 B tasks afterwards in the queue, then we want to run 5 type A and 5 type B side by side regardless of the order they were queued in.

      Suppose you only have 1 ad hoc task runners and 500 A and then 500 B tasks afterwards in the queue, then we want to run 1 type A and then 1 type B and then 1 type A regardless of the order they were queued in.

       The intention behind MDL-64610 is more or less similar but the approach is quite different.

       

       

       

            brendanheywood Brendan Heywood
            brendanheywood Brendan Heywood
            Dmitrii Metelkin Dmitrii Metelkin
            Sara Arjona (@sarjona) Sara Arjona (@sarjona)
            Gladys Basiana Gladys Basiana
            Votes:
            1 Vote for this issue
            Watchers:
            10 Start watching this issue

              Created:
              Updated:
              Resolved:

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0 minutes
                0m
                Logged:
                Time Spent - 2 hours, 40 minutes
                2h 40m

                  Error rendering 'clockify-timesheets-time-tracking-reports:timer-sidebar'. Please contact your Jira administrators.