Moodle

Large number of quiz attempts (effectively) break backup

Details

  • Type: Bug Bug
  • Status: Closed Closed
  • Priority: Major Major
  • Resolution: Duplicate
  • Affects Version/s: 1.6.5
  • Fix Version/s: None
  • Component/s: Backup, Quiz
  • Labels:
    None
  • Affected Branches:
    MOODLE_16_STABLE

Description

I've been unable to verify if this problem persists in newer versions that 1.6.5 - but I should be able to soon

If a course has a large (but not ridiculous) number of quiz attempts (order of 1000 for a quiz with about 25 questions) backup and/or restore is severely compromised. It seems that building the XML for those attempts burns up crazy amounts of memory (we went up to half a gig before giving up). A similar problem is exhibited trying to restore the file - this has been verified as a problem in 1.8 STABLE.

IMHO, backup and restore shouldn't ever use that amount of memory, although I appreciate the solution will be complex.

Issue Links

This issue will be resolved by:
MDL-15489 Restore fails: Could not restore modules! Critical Open
MDL-14269 META: Backup performance Minor Closed
This issue has a non-specific relationship to:
MDL-14302 Course Restore Fails with Heavy User Data Minor Closed

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Eloy Lafuente (stronk7) added a comment -

Hi Howard,

the most important memory eater in backup and restore is, AFAIK, the zip and unzip libraries/executables. If you are using the internal ones... can you try with the external executables instead?

the numbers you expose above (1000 attempts with 25 questions each) doesn't sound really high. It makes, more or less, 25000 answers to be stored and, using 1K by answer (and that's an exaggerated measure) it's around 25MB only (far from the half a gig).

Can you confirm the zip/unzip thing? If the problem persists we can do further tests to see if something else is happening.

TIA and ciao

Show
Eloy Lafuente (stronk7) added a comment - Hi Howard, the most important memory eater in backup and restore is, AFAIK, the zip and unzip libraries/executables. If you are using the internal ones... can you try with the external executables instead? the numbers you expose above (1000 attempts with 25 questions each) doesn't sound really high. It makes, more or less, 25000 answers to be stored and, using 1K by answer (and that's an exaggerated measure) it's around 25MB only (far from the half a gig). Can you confirm the zip/unzip thing? If the problem persists we can do further tests to see if something else is happening. TIA and ciao
Hide
Permalink
Howard Miller added a comment -

Martin,

Unfortunately, I no longer have the sites available to test as the only way around the problem was to backup without the quiz attempts. However, I recall that I did try it at the time with the external executables and although I did relieve the problem slightly it still did not complete the backup. However, I did encounter the problem on a number of courses and sites so I am reasonably confident the problem exists.

I'll have a think about how I can generate some test data to demonstrate this (or at least see if it persists on 1.8).

Show
Howard Miller added a comment - Martin, Unfortunately, I no longer have the sites available to test as the only way around the problem was to backup without the quiz attempts. However, I recall that I did try it at the time with the external executables and although I did relieve the problem slightly it still did not complete the backup. However, I did encounter the problem on a number of courses and sites so I am reasonably confident the problem exists. I'll have a think about how I can generate some test data to demonstrate this (or at least see if it persists on 1.8).
Hide
Permalink
Tim Hunt added a comment -

Neither mod/quiz/backuplib.php nor question/backuplib seem to be doing anything too evil.

$quiz_attempts = get_records("quiz_attempts","quiz",$quiz,"id");

$question_states = get_records("question_states","attempt",$attempt,"id");

$rqp_state = get_record("question_rqp_states","stateid",$state);

so it loads one record about each attempt into memory. Then for each attempt, it loads all the records about that attempt.

So, while these would be prime candidates for get_recordSET, I don't think that is what is causing memory to run out.

Show
Tim Hunt added a comment - Neither mod/quiz/backuplib.php nor question/backuplib seem to be doing anything too evil. $quiz_attempts = get_records("quiz_attempts","quiz",$quiz,"id"); $question_states = get_records("question_states","attempt",$attempt,"id"); $rqp_state = get_record("question_rqp_states","stateid",$state); so it loads one record about each attempt into memory. Then for each attempt, it loads all the records about that attempt. So, while these would be prime candidates for get_recordSET, I don't think that is what is causing memory to run out.
Hide
Permalink
Dan Marsden added a comment -

Howard - any idea if this is still an issue with newer versions of Moodle?

Dan

Show
Dan Marsden added a comment - Howard - any idea if this is still an issue with newer versions of Moodle? Dan
Hide
Permalink
Howard Miller added a comment -

Definitely still an issue in 1.8.2 - it can really burn up memory.

Show
Howard Miller added a comment - Definitely still an issue in 1.8.2 - it can really burn up memory.
Hide
Permalink
Eloy Lafuente (stronk7) added a comment -

What if we close this and try to advance in MDL-14302 (seems to be the same problem, but under 1.9).

Ciao

Show
Eloy Lafuente (stronk7) added a comment - What if we close this and try to advance in MDL-14302 (seems to be the same problem, but under 1.9). Ciao
Hide
Permalink
Dan Marsden added a comment -

MDL-14302 is related to non-english characters in the xml - Howard - do you have any non-english characters used in any of your courses?

thanks,

Dan

Show
Dan Marsden added a comment - MDL-14302 is related to non-english characters in the xml - Howard - do you have any non-english characters used in any of your courses? thanks, Dan
Hide
Permalink
Howard Miller added a comment -

I honestly don't know. The main problem has been in our School of Dentistry. They heavily use quizzes but I doubt they use non-english characters.

Show
Howard Miller added a comment - I honestly don't know. The main problem has been in our School of Dentistry. They heavily use quizzes but I doubt they use non-english characters.
Hide
Permalink
Eloy Lafuente (stronk7) added a comment -

I'm closing this as duplicate of MDL-15489. Update your watches.

Show
Eloy Lafuente (stronk7) added a comment - I'm closing this as duplicate of MDL-15489. Update your watches.

People

Vote (2)
Watch (3)

Dates

  • Created:
    Updated:
    Resolved:
Atlassian JIRA (v4.4.3#663-r165197) | Report a problem
Powered by a free Atlassian JIRA open source license for Moodle. Try JIRA - bug tracking software for your team.