Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-79254

Quiz statistics processing task doesn't support large processing loads

XMLWordPrintable

    • MOODLE_402_STABLE
    • MOODLE_401_STABLE, MOODLE_402_STABLE
    • MDL-79254_401_STABLE
    • MDL-79254_master
    • Hide

      Queueing the task in response to a quiz attempt is covered by the new attempt_submitted_test class. The actual calculation performed by the task is already covered in statistics_bulk_loader_test.

      1. Create a quiz, on a course with at least 2 students.
      2. From the CLI, run `php admin/cli/adhoc_task.php --execute --classname="\quiz_statistics\task\recalculate"` to run any queued recalculation tasks.
        • You should see "Ran 0 adhoc tasks" and no output from the recalculate task.
      3. Log in as each student, start and finish a quiz attempt.
      4. From the CLI, run `php admin/cli/adhoc_task.php --execute --classname="\quiz_statistics\task\recalculate"` again.
        • You should see "Ran 0 adhoc tasks" and no output from the recalculate task.
      5. Wait 1 hour (or fiddle with the system time to make it appear as though it is an hour later).
      6. From the CLI, run `php admin/cli/adhoc_task.php --execute --classname="\quiz_statistics\task\recalculate"` again.
        • You should see "Re-calculating statistics for quiz {quizname} ({quizid}) from course {shortname} ({courseid}) with 2 attempts".
        • You should see "Ran 1 adhoc tasks".
      Show
      Queueing the task in response to a quiz attempt is covered by the new attempt_submitted_test class. The actual calculation performed by the task is already covered in statistics_bulk_loader_test. Create a quiz, on a course with at least 2 students. From the CLI, run `php admin/cli/adhoc_task.php --execute --classname="\quiz_statistics\task\recalculate"` to run any queued recalculation tasks. You should see "Ran 0 adhoc tasks" and no output from the recalculate task. Log in as each student, start and finish a quiz attempt. From the CLI, run `php admin/cli/adhoc_task.php --execute --classname="\quiz_statistics\task\recalculate"` again. You should see "Ran 0 adhoc tasks" and no output from the recalculate task. Wait 1 hour (or fiddle with the system time to make it appear as though it is an hour later). From the CLI, run `php admin/cli/adhoc_task.php --execute --classname="\quiz_statistics\task\recalculate"` again. You should see "Re-calculating statistics for quiz {quizname} ({quizid}) from course {shortname} ({courseid}) with 2 attempts". You should see "Ran 1 adhoc tasks".

      https://github.com/moodle/moodle/blob/master/mod/quiz/report/statistics/classes/task/recalculate.php#L75-L79

      Currently this is hardcoded to kill the task after it's been running for an hour.

      This reports back as task failed rather than task succesful which then introduces a fail delay. 

      I'm not clear why it's using this arbitrary 1h value rather than $CFG->task_scheduled_max_runtime if anything.

       

      But if this is to avoid hogging the task processing, that's fine to break up in parts but really needs to be able to catchup with itself without being marked as failed.

       

      Clearly 1h will be far too low for a big site like ours.

       

      While there was a stack trace along the way this doesn't seem to be the reason the task ended but maybe it's causing of the false positive failed status

      42424bf30553:1092914 This task has been running for more than 1 hour so stopping this execution.
      		 
      42424bf30553:1092914 ... used 335926 dbqueries
      		 
      42424bf30553:1092914 ... used 4345.8472127914 seconds
      		 
      42424bf30553:1092914 Scheduled task failed: Recalculate question statistics (quiz_statistics\task\recalculate),Coding error detected, it must be fixed by a programmer: moodle_database::get_in_or_equal() does not accept empty arrays
      		 
      42424bf30553:1092914 Backtrace:
      		 
      42424bf30553:1092914 * line 564 of /question/engine/datalib.php: call to moodle_database->get_in_or_equal()
      		 
      42424bf30553:1092914 * line 295 of /question/classes/statistics/questions/calculator.php: call to question_engine_data_mapper->load_questions_usages_latest_steps()
      		 
      42424bf30553:1092914 * line 96 of /question/classes/statistics/questions/calculator.php: call to core_question\statistics\questions\calculator->get_latest_steps()
      		 
      42424bf30553:1092914 * line 654 of /mod/quiz/report/statistics/report.php: call to core_question\statistics\questions\calculator->calculate()
      		 
      42424bf30553:1092914 * line 945 of /mod/quiz/report/statistics/report.php: call to quiz_statistics_report->get_all_stats_and_analysis()
      		 
      42424bf30553:1092914 * line 108 of /mod/quiz/report/statistics/classes/task/recalculate.php: call to quiz_statistics_report->calculate_questions_stats_for_question_bank()
      		 
      42424bf30553:1092914 * line 405 of /lib/classes/cron.php: call to quiz_statistics\task\recalculate->execute()
      		 
      42424bf30553:1092914 * line 208 of /lib/classes/cron.php: call to core\cron::run_inner_scheduled_task()
      		 
      42424bf30553:1092914 * line 125 of /lib/classes/cron.php: call to core\cron::run_scheduled_tasks()
      		 
      42424bf30553:1092914 * line 186 of /admin/cli/cron.php: call to core\cron::run_main_process()
      		 
      42424bf30553:1092914

       

       

        1. (1) 2 Passed -- (Master)MDL-79254.png
          (1) 2 Passed -- (Master)MDL-79254.png
          29 kB
        2. (1) 6 Passed -- (Master)MDL-79254.png
          (1) 6 Passed -- (Master)MDL-79254.png
          73 kB

            marxjohnson Mark Johnson
            aspark Alistair Spark
            Dale Davies Dale Davies
            Tim Hunt Tim Hunt
            Kim Jared Lucas Kim Jared Lucas
            Votes:
            4 Vote for this issue
            Watchers:
            14 Start watching this issue

              Created:
              Updated:
              Resolved:

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0 minutes
                0m
                Logged:
                Time Spent - 6 hours, 15 minutes
                6h 15m

                  Error rendering 'clockify-timesheets-time-tracking-reports:timer-sidebar'. Please contact your Jira administrators.