Moodle

group member data in Backups is repeated endlessly

Details

  • Type: Bug Bug
  • Status: Closed Closed
  • Priority: Major Major
  • Resolution: Fixed
  • Affects Version/s: 1.8.6, 1.9.2
  • Fix Version/s: 1.8.7, 1.9.3
  • Component/s: Backup
  • Labels:
    None
  • Environment:
    All os's and hardware
  • Affected Branches:
    MOODLE_18_STABLE, MOODLE_19_STABLE
  • Fixed Branches:
    MOODLE_18_STABLE, MOODLE_19_STABLE

Description

All backups will include group info. the group members info included is repeated too many times making the moodle.xml file too large to restore in extreme cases. The number repeats seems to depend on the number of students enrolled on the course and the number of groups on the course.

This problem is present at the OU, I tracked it down to a line of sql in in backuplib.php in the method backup_groups_members_info() that reads

The sql reads as :
SELECT gm.* FROM pssr1_mdl_groups_members gm, pssr1_mdl_backup_ids bi WHERE gm.groupid = 44 AND bi.backup_code = 1220629007 AND bi.table_name = 'user'

The problem is serious because I have a course with a large number of students enrolled and asmany as 314 groups that cannot be restored. Even generating a backup with just labels and nothing else, no user data or anything generates a 35mb zip which contains a 374mb moodle.xml file. 95% of this file is made up of repeating group member data.

I made a test course with only a couple of students and groups and found that the sql listed above determined the results set from the db which determined the xml output.

I found a simple fix was to added the line

AND bi.old_id = gm.userid

to the end of the sql to restrict the id's correctly. I checked moodle core and this bug appears in there too so I wanted to check if a) this is the correct fix since an even simpler solution could be to remove the references to the 'mdl_backup_ids' table and make cleaner sql. I also wanted to know if others can replicate the problem.

I'm also wondering why a join wasn't used? Is it because the data access layer can't support it?

Activity

Hide
Eloy Lafuente (stronk7) added a comment -

Hi Colin,

well spotted!

The JOIN condition is 100% missing there, I'll add it now!

Anyway, just for curiosity.... that query shouldn't be returning duplicate records at all. It's one get_records() call, and, as it returns one associative (by id) array, php should be "removing" duplicates transparently. In fact, I've executed it here and cannot get duplicates from within Moodle (but I get them from line command, of course). How can you be getting such duplicates there?

Ciao

Show
Eloy Lafuente (stronk7) added a comment - Hi Colin, well spotted! The JOIN condition is 100% missing there, I'll add it now! Anyway, just for curiosity.... that query shouldn't be returning duplicate records at all. It's one get_records() call, and, as it returns one associative (by id) array, php should be "removing" duplicates transparently. In fact, I've executed it here and cannot get duplicates from within Moodle (but I get them from line command, of course). How can you be getting such duplicates there? Ciao
Hide
Eloy Lafuente (stronk7) added a comment -

Missing condition has been added to 18_STABLE, 19_STABLE and HEAD. Resolving as fixed. Thanks!

Show
Eloy Lafuente (stronk7) added a comment - Missing condition has been added to 18_STABLE, 19_STABLE and HEAD. Resolving as fixed. Thanks!
Hide
Colin Chambers added a comment -

Hi eloy,

thanks for the prompt response. I didn't know the php methods should be removing the duplicates so yes I'm surprised it didn't. What I noticed was that the output of members info into the backup xml matched exactly the recordset returned from the sql query. so it was obvious the sql needed fixing. That's all I can say.

It was on my mind that there are difference in our moodle due to the work we've done which could mean the contents of either table is subtly different and causing this issue. In this particular method however the sql is as it is in moodle head so I'm stumped why it's different.

Maybe you'll find the problem if you replicate (partly, not with as many users I expect) the set up of the course with the problem. basically around 300 groups had around 1000 students in those groups.

I mimicked this with just 8 students enrolled and 2 groups. the problem was evident in the backup file straight away. The xml file grew or shrunk depending on the number of students on the course but by a large amount not just one record.

Any way it appears fixed now after testing. thanks

Colin

Show
Colin Chambers added a comment - Hi eloy, thanks for the prompt response. I didn't know the php methods should be removing the duplicates so yes I'm surprised it didn't. What I noticed was that the output of members info into the backup xml matched exactly the recordset returned from the sql query. so it was obvious the sql needed fixing. That's all I can say. It was on my mind that there are difference in our moodle due to the work we've done which could mean the contents of either table is subtly different and causing this issue. In this particular method however the sql is as it is in moodle head so I'm stumped why it's different. Maybe you'll find the problem if you replicate (partly, not with as many users I expect) the set up of the course with the problem. basically around 300 groups had around 1000 students in those groups. I mimicked this with just 8 students enrolled and 2 groups. the problem was evident in the backup file straight away. The xml file grew or shrunk depending on the number of students on the course but by a large amount not just one record. Any way it appears fixed now after testing. thanks Colin

People

Vote (0)
Watch (0)

Dates

  • Created:
    Updated:
    Resolved: