Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-47271

Add system status report including tasks with cli status script

    XMLWordPrintable

Details

    • MOODLE_27_STABLE, MOODLE_33_STABLE, MOODLE_39_STABLE
    • MOODLE_39_STABLE
    • MDL-47271-status-api-fixes
    • Hide

      1) Visit the new report page: /report/status/index.php

      On a properly running site confirm everything is green

      2) Run the cli and confirm it is OK

      php admin/cli/checks.php

      OK: All 'status' checks ok

      Confirm that the shell error code was 0:

      $ echo $?
      0

      3) Run it in verbose mode

       

      Lets break everything

      4) Break environment check, edit admin/environment.xml, find the moodle version 3.9 php version required (line 2998) and change it to php version 17, confirm it is an error:

      Now undo this.

      5) Simulate a pending upgrade, bump the version.php and confirm the error:

      Also confirm that the shell error code was 2 which is an NRPE critical error:

      $ echo $?
      2

      6) Turn cron off. Now simulate it having never run and confirm the error:

      php admin/cli/cfg.php --component=tool_task --name=lastcronstart --unset

      7) Simulate it having run 40 hours ago and confirm an error:

       

      $ php admin/cli/cfg.php --component=tool_task --name=lastcronstart --set=`date -d '40 hour ago' +'%s'`
      $ php admin/cli/checks.php
       CRITICAL: Cron running (tool_task_cronrunning)
          Status | Check
       ----------+--------------------------------------------------------------------
       CRITICAL | Cron running (tool_task_cronrunning) | The admin/cron.php script has not been run for 1 day 16 | hours and should run every 1 min.
      

       

      8) Simulate it having run 1 hour ago and confirm an warning

       

      $ php admin/cli/cfg.php --component=tool_task --name=lastcronstart --set=`date -d '1 hour ago' +'%s'`
      $ php admin/cli/checks.php
      WARNING: Cron running (tool_task_cronrunning)
         Status | Check                                                       
      ----------+--------------------------------------------------------------------
        WARNING | Cron running (tool_task_cronrunning)                        
                |     The admin/cron.php script has not been run for 1 hour and
                |     should run every 1 min.
      

       Also confirm that the shell error code was 1 which is an NRPE warning:

      $ echo $?
      1

       9) Make it good again, and confirm it is ok

       

      $ php admin/cli/cfg.php --component=tool_task --name=lastcronstart --set=`date -d '1 sec ago' +'%s'`
      $ php admin/cli/checks.php
      OK: All 'status' checks ok
      
      

       10) Now simulate gaps between cron 

       

      $ php admin/cli/cfg.php --component=tool_task --name=lastcroninterval --set=600
      $ php admin/cli/checks.php 
      WARNING: Cron running (tool_task_cronrunning)
         Status | Check                                                       
      ----------+--------------------------------------------------------------------
        WARNING | Cron running (tool_task_cronrunning)                        
                |     There was 10 mins between the last two runs of the cron
                |     maintenance script and it should run every 1 min. We
                |     recommend configuring it to run more frequently.
      
      

       10) Install https://github.com/catalyst/moodle-tool_testtasks

      11) Queue up some tasks:

      php admin/tool/testtasks/cli/queue_adhoc_tasks.php

      12) Confirm these appear as INFO:

      $ php admin/cli/checks.php -v --filter=adhoc

       

      13) Drop into sql and fudge the dates to be slightly old and confirm you get a warning:

      update mdl_task_adhoc set nextruntime = round(extract(epoch from now()) - 60 * 10);

       

      14) Fudge them to be older and confirm you get a warning

      update mdl_task_adhoc set nextruntime = round(extract(epoch from now()) - 60 * 60);

      15) Update the warning and error threshold and retest:

      php admin/cli/cfg.php --name=adhoctaskagewarn --set=500

      php admin/cli/cfg.php --name=adhoctaskageerror --set=600

       

       

      Show
      1) Visit the new report page: /report/status/index.php On a properly running site confirm everything is green 2) Run the cli and confirm it is OK php admin/cli/checks.php OK: All 'status' checks ok Confirm that the shell error code was 0: $ echo $? 0 3) Run it in verbose mode   Lets break everything 4) Break environment check, edit admin/environment.xml, find the moodle version 3.9 php version required (line 2998) and change it to php version 17, confirm it is an error: Now undo this. 5) Simulate a pending upgrade, bump the version.php and confirm the error: Also confirm that the shell error code was 2 which is an NRPE critical error: $ echo $? 2 6) Turn cron off. Now simulate it having never run and confirm the error: php admin/cli/cfg.php --component=tool_task --name=lastcronstart --unset 7) Simulate it having run 40 hours ago and confirm an error:   $ php admin/cli/cfg.php --component=tool_task --name=lastcronstart --set=`date -d '40 hour ago' + '%s' ` $ php admin/cli/checks.php CRITICAL: Cron running (tool_task_cronrunning) Status | Check ----------+-------------------------------------------------------------------- CRITICAL | Cron running (tool_task_cronrunning) | The admin/cron.php script has not been run for 1 day 16 | hours and should run every 1 min.   8) Simulate it having run 1 hour ago and confirm an warning   $ php admin/cli/cfg.php --component=tool_task --name=lastcronstart --set=`date -d '1 hour ago' +'%s'` $ php admin/cli/checks.php WARNING: Cron running (tool_task_cronrunning) Status | Check ----------+-------------------------------------------------------------------- WARNING | Cron running (tool_task_cronrunning) | The admin/cron.php script has not been run for 1 hour and | should run every 1 min.  Also confirm that the shell error code was 1 which is an NRPE warning: $ echo $? 1  9) Make it good again, and confirm it is ok   $ php admin/cli/cfg.php --component=tool_task --name=lastcronstart --set=`date -d '1 sec ago' +'%s'` $ php admin/cli/checks.php OK: All 'status' checks ok  10) Now simulate gaps between cron    $ php admin/cli/cfg.php --component=tool_task --name=lastcroninterval --set= 600 $ php admin/cli/checks.php WARNING: Cron running (tool_task_cronrunning) Status | Check ----------+-------------------------------------------------------------------- WARNING | Cron running (tool_task_cronrunning) | There was 10 mins between the last two runs of the cron | maintenance script and it should run every 1 min. We | recommend configuring it to run more frequently.  10) Install  https://github.com/catalyst/moodle-tool_testtasks 11) Queue up some tasks: php admin/tool/testtasks/cli/queue_adhoc_tasks.php 12) Confirm these appear as INFO: $ php admin/cli/checks.php -v --filter=adhoc   13) Drop into sql and fudge the dates to be slightly old and confirm you get a warning: update mdl_task_adhoc set nextruntime = round(extract(epoch from now()) - 60 * 10);   14) Fudge them to be older and confirm you get a warning update mdl_task_adhoc set nextruntime = round(extract(epoch from now()) - 60 * 60); 15) Update the warning and error threshold and retest: php admin/cli/cfg.php --name=adhoctaskagewarn --set=500 php admin/cli/cfg.php --name=adhoctaskageerror --set=600    

    Description

      For the purposes of monitoring, it is good to have a CLI script which can be run to verify the status of scheduled tasks.

      It could report problems such as:

      • Tasks which have been scheduled to run but have not been run recently
      • Tasks with faildelay present

      Taking parameters for tolerances and reporting 0 for no problems.

      Attachments

        Issue Links

          Activity

            People

              brendanheywood Brendan Heywood
              poltawski Dan Poltawski
              Peter Burnett Peter Burnett
              Andrew Lyons Andrew Lyons
              Janelle Barcega Janelle Barcega
              Andrew Lyons, Huong Nguyen, Jun Pataleta, Michael Hawkins, Shamim Rezaie, Simey Lameze, Amaia Anabitarte, Carlos Escobedo, Ferran Recio, Ilya Tregubov, Sara Arjona (@sarjona), Amaia Anabitarte, Carlos Escobedo, Ferran Recio, Ilya Tregubov, Sara Arjona (@sarjona)
              Votes:
              15 Vote for this issue
              Watchers:
              23 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:
                15/Jun/20

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 1 hour, 36 minutes
                  1h 36m