Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-79293

Timeout when viewing question bank on a large InnoDB table

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Duplicate
    • Icon: Minor Minor
    • None
    • 4.1.5
    • Questions
    • None
    • MOODLE_401_STABLE

      DELETE queries on InnoDB tables with a large number of records are really slow and introduce timeouts on our Moodle installation (currently on 4.1.5).
      A fix is needed for the question/classes/statistics/responses/analysis_for_question.php:cache function, as is implemented in question/engine/datalib.php:1011: delete_usage_records_for_mysql, where the deletes are chunked into pieces of 1000 (though, chunk size of 10000 introduced no noticable overhead and reduced iterations by a factor of 10).

      Should this issue be addressed in question/classes/statistics/responses/analysis_for_question.php, or would it be sensible to modify $DB->delete_records method directly for mysql and mariadb database types?

      We have 900.000 records in question_response_analysis table. Without this fix, the page returned a timeout as the query usually took more than 300s. With the fix, the page renders normally after 3s.

      Our fix was to change the cache function to:

          /**
           * Save the analysis to the DB, first cleaning up any old ones.
           *
           * @param \qubaid_condition $qubaids    which question usages have been analysed.
           * @param string            $whichtries which tries have been analysed?
           * @param int               $questionid which question.
           * @param int|null          $calculationtime time when the analysis was done. (Defaults to time()).
           */
          public function cache($qubaids, $whichtries, $questionid, $calculationtime = null) {
              global $DB;
       
              $transaction = $DB->start_delegated_transaction();
       
              // Delete in chunks for better performance on InnoDB
              // https://git.arnes.si/ucilnice/moodle/-/issues/12
              $allids = $DB->get_records_sql_menu("
                      SELECT DISTINCT id, id AS id2
                        FROM {question_response_analysis}
                      WHERE hashcode = ? AND whichtries = ? AND questionid = ?",
                      [$qubaids->get_hash_code(), $whichtries, $questionid]
              );
              foreach (array_chunk($allids, 10000) as $todelete) {
                  list($idsql, $idparams) = $DB->get_in_or_equal($todelete);
                  $DB->execute('
                          DELETE FROM {question_response_count}
                           WHERE id ' . $idsql,
                           $idparams
                  );
                  $DB->execute('
                          DELETE FROM {question_response_analysis}
                           WHERE id ' . $idsql,
                           $idparams
                   );
              }
       
              foreach ($this->get_variant_nos() as $variantno) {
                  foreach ($this->get_subpart_ids($variantno) as $subpartid) {
                      $analysisforsubpart = $this->get_analysis_for_subpart($variantno, $subpartid);
                      $analysisforsubpart->cache($qubaids, $whichtries, $questionid, $variantno, $subpartid, $calculationtime);
                  }
              }
       
              $transaction->allow_commit();
          }
       
      
      

       

      Related issue: /MDL-50798

      Does anyone else with a large Moodle installation have this problem? Should I submit a PR with the cache function fix, or should this be handled specifically for mysql/mariadb?

            Unassigned Unassigned
            tjazbec Timotej Jazbec
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved:

                Error rendering 'clockify-timesheets-time-tracking-reports:timer-sidebar'. Please contact your Jira administrators.