-
Bug
-
Resolution: Duplicate
-
Minor
-
None
-
4.1.5
-
None
-
MOODLE_401_STABLE
DELETE queries on InnoDB tables with a large number of records are really slow and introduce timeouts on our Moodle installation (currently on 4.1.5).
A fix is needed for the question/classes/statistics/responses/analysis_for_question.php:cache function, as is implemented in question/engine/datalib.php:1011: delete_usage_records_for_mysql, where the deletes are chunked into pieces of 1000 (though, chunk size of 10000 introduced no noticable overhead and reduced iterations by a factor of 10).
Should this issue be addressed in question/classes/statistics/responses/analysis_for_question.php, or would it be sensible to modify $DB->delete_records method directly for mysql and mariadb database types?
We have 900.000 records in question_response_analysis table. Without this fix, the page returned a timeout as the query usually took more than 300s. With the fix, the page renders normally after 3s.
Our fix was to change the cache function to:
/** |
* Save the analysis to the DB, first cleaning up any old ones.
|
*
|
* @param \qubaid_condition $qubaids which question usages have been analysed.
|
* @param string $whichtries which tries have been analysed?
|
* @param int $questionid which question.
|
* @param int|null $calculationtime time when the analysis was done. (Defaults to time()).
|
*/
|
public function cache($qubaids, $whichtries, $questionid, $calculationtime = null) { |
global $DB;
|
|
$transaction = $DB->start_delegated_transaction();
|
|
// Delete in chunks for better performance on InnoDB |
// https://git.arnes.si/ucilnice/moodle/-/issues/12 |
$allids = $DB->get_records_sql_menu("
|
SELECT DISTINCT id, id AS id2
|
FROM {question_response_analysis}
|
WHERE hashcode = ? AND whichtries = ? AND questionid = ?",
|
[$qubaids->get_hash_code(), $whichtries, $questionid]
|
);
|
foreach (array_chunk($allids, 10000) as $todelete) { |
list($idsql, $idparams) = $DB->get_in_or_equal($todelete);
|
$DB->execute('
|
DELETE FROM {question_response_count}
|
WHERE id ' . $idsql,
|
$idparams
|
);
|
$DB->execute('
|
DELETE FROM {question_response_analysis}
|
WHERE id ' . $idsql,
|
$idparams
|
);
|
}
|
|
foreach ($this->get_variant_nos() as $variantno) { |
foreach ($this->get_subpart_ids($variantno) as $subpartid) { |
$analysisforsubpart = $this->get_analysis_for_subpart($variantno, $subpartid); |
$analysisforsubpart->cache($qubaids, $whichtries, $questionid, $variantno, $subpartid, $calculationtime);
|
}
|
}
|
|
$transaction->allow_commit();
|
}
|
|
|
Related issue: /MDL-50798
Does anyone else with a large Moodle installation have this problem? Should I submit a PR with the cache function fix, or should this be handled specifically for mysql/mariadb?