Issue Details (XML | Word | Printable)

Key: MDL-20116
Type: Bug Bug
Status: Open Open
Priority: Critical Critical
Assignee: Eloy Lafuente (stronk7)
Reporter: Jeffrey Silverman
Votes: 1
Watchers: 2
Operations

Add/Edit UI Mockup to this issue
If you were logged in you would be able to see more operations.
Moodle

Backups take a long time due to too small chunk size in backuplib.php function backup_log_info

Created: 18/Aug/09 06:15 PM   Updated: 18/Aug/09 06:15 PM
Return to search
Component/s: Backup
Affects Version/s: 1.9.4, 1.9.5
Fix Version/s: None

Environment: Solaris 10, Mysql 5.0.54, PHP 5.2.4

Participants: Eloy Lafuente (stronk7) and Jeffrey Silverman
Security Level: None
Affected Branches: MOODLE_19_STABLE


 Description  « Hide
We have struggled with cron.php's automatic backups taking an inordinately long time. Like, many, MANY hours (40+ for one site).

There may be more than one thing going on here, but one performance improvement I made that helped dramatically was increasing the chunk size tenfold in backuplib.php function backup_log_info():

Snippet lines 1463-1468
-----
    function backup_log_info($bf,$preferences) {

        global $CFG;

        //Number of records to get in every chunk
        // $recordset_size = 1000;
        // CHANGE THIS to 10k:
        $recordset_size = 10000;
-----

I tried upping it to 100k but that did not significantly further improve performance, and also required an increase in PHP's max_memory configuration.

For courses with large logs -- "Front Page" for example -- there were hundreds of chunks. All these database calls were very time consuming.

I strongly recommend increasing the chunk size here. I hate to have to locally patch our builds for little things like this, so it would be great to see this pushed back to Moodle core.

Any comments on this topic are welcome! thanks

 All   Comments   Change History   Version Control      Sort Order: Ascending order - Click to sort in descending order
No changes have yet been made on this issue.