Moodle

assignment submissions can contain duplicates which causes problems during upgrade to new gradebook

Details

  • Type: Bug Bug
  • Status: Closed Closed
  • Priority: Blocker Blocker
  • Resolution: Fixed
  • Affects Version/s: 1.9
  • Fix Version/s: 1.9, 2.0
  • Component/s: Assignment, Gradebook
  • Labels:
    None
  • Environment:
    Debian Linux, PHP 4.4.4.8.4, MySQL 5.0.27
  • Database:
    MySQL
  • Affected Branches:
    MOODLE_19_STABLE
  • Fixed Branches:
    MOODLE_19_STABLE, MOODLE_20_STABLE

Description

Testing upgrade from Moodle 1.8.2 (2007021520) to 1.9 Beta 2 (2007101500), get the error 'Found more than one record in fetch() !' when attempting to set up Assignment tables. I've attached the upgrade file.

Upgrade halts at this point and cannot proceed.

  1. grade_object.php
    07/Nov/07 12:17 AM
    13 kB
    Karlene Clapp
  2. upg_20071025-1138.html
    26/Oct/07 12:53 AM
    0.3 kB
    Matt Campbell

Activity

Hide
Matt Campbell added a comment -

I turned debug on in /mod/assignment/upgrade.php and started this again, now get:

SELECT * FROM mdl_grade_grades WHERE itemid = '191' AND userid = '3751' ORDER BY id
Found more than one record in fetch() !

When I pull up mdl_grade_grades, I find two entries that meet this criteria.

I deleted the entry with the higher id and started again, got the error again - same itemid, different userid. Went into database and found several duplicates for itemid = 191, deleted the second entry on each duplicate.

Ran again, same error this time, but itemid = 190.

Continuing process - I'll comment back how it goes.

The only extra factor I can think of in this is that we ARE using GBPV2.

Show
Matt Campbell added a comment - I turned debug on in /mod/assignment/upgrade.php and started this again, now get: SELECT * FROM mdl_grade_grades WHERE itemid = '191' AND userid = '3751' ORDER BY id Found more than one record in fetch() ! When I pull up mdl_grade_grades, I find two entries that meet this criteria. I deleted the entry with the higher id and started again, got the error again - same itemid, different userid. Went into database and found several duplicates for itemid = 191, deleted the second entry on each duplicate. Ran again, same error this time, but itemid = 190. Continuing process - I'll comment back how it goes. The only extra factor I can think of in this is that we ARE using GBPV2.
Hide
Matt Campbell added a comment -

Only three courses out of over 700 had this issue, couldn't seem to find any factor that was unique just to them. Once I'd gone through all the mdl_grade_grades issues with the assignments, installation continued normally.

Show
Matt Campbell added a comment - Only three courses out of over 700 had this issue, couldn't seem to find any factor that was unique just to them. Once I'd gone through all the mdl_grade_grades issues with the assignments, installation continued normally.
Hide
Yu Zhang added a comment -

Hi Matt,

Are you able to provide a copy of your db before the upgrade to 1.9? That would help us to locate the problem... Also, are all the affected grade items assignments?

Cheers,

Yu

Show
Yu Zhang added a comment - Hi Matt, Are you able to provide a copy of your db before the upgrade to 1.9? That would help us to locate the problem... Also, are all the affected grade items assignments? Cheers, Yu
Hide
Matt Campbell added a comment -

Problem was ONLY with assignments, all other graded activities went through just fine.

I've sent you an email for the copy of the database - please confirm that you have gotten the file.

Thanks,
Matt

Show
Matt Campbell added a comment - Problem was ONLY with assignments, all other graded activities went through just fine. I've sent you an email for the copy of the database - please confirm that you have gotten the file. Thanks, Matt
Hide
Yu Zhang added a comment -

Hi Matt, got your email, will take a look, cheers, Yu

Show
Yu Zhang added a comment - Hi Matt, got your email, will take a look, cheers, Yu
Hide
Yu Zhang added a comment -

I think most likely we are getting this from more than 1 record in assignment_submissions for the same assignment for the same user. This goes directly into grade_grades during upgrade. I had a similar report earlier, problem was in assignment as well. Currently there is no unique constraint for assignment_submissions nor grade_grades table. Should we at least enforce it in grade_grades db? And what should we do with Moodle 1.8 coming with multiple grades?

Show
Yu Zhang added a comment - I think most likely we are getting this from more than 1 record in assignment_submissions for the same assignment for the same user. This goes directly into grade_grades during upgrade. I had a similar report earlier, problem was in assignment as well. Currently there is no unique constraint for assignment_submissions nor grade_grades table. Should we at least enforce it in grade_grades db? And what should we do with Moodle 1.8 coming with multiple grades?
Hide
Petr Škoda (skodak) added a comment -

Thanks for all info, I will try to add some code to prevent this type of problems...

Show
Petr Škoda (skodak) added a comment - Thanks for all info, I will try to add some code to prevent this type of problems...
Hide
Martin Dougiamas added a comment -

There definitely should be a formal constraint on grade_grades ... Yu,can you add that?

Petr, can you look into the upgrade process and make it deal better with duplicates?

Show
Martin Dougiamas added a comment - There definitely should be a formal constraint on grade_grades ... Yu,can you add that? Petr, can you look into the upgrade process and make it deal better with duplicates?
Hide
Petr Škoda (skodak) added a comment -

assigning back to Yu, I can not find anything wrong and do not have data to replicate, thanks

Show
Petr Škoda (skodak) added a comment - assigning back to Yu, I can not find anything wrong and do not have data to replicate, thanks
Hide
Karlene Clapp added a comment -

I had this exact issue and after manually deleting several of the duplicates, I wrote a patch in the lib/grade/grade_object.php file to delete the duplicates as it finds them. It's kind of a hack, but it has worked each time I've done an upgrade and prevents the constant stopping during the assignment module upgrade. I have attached the file. Each line I added has a comment with "KC".

Note: I used a global variable to store the $sql, but I'm sure there's a better way. This was just a quick fix, but you get the idea.

These are the two functions that were changed:

/**

  • Factory method - uses the parameters to retrieve matching instance from the DB.
  • @static final protected
  • @return mixed object insatnce or false if not found
    */
    function fetch_helper($table, $classname, $params) {
    // we have to do use this hack because of the incomplete OOP implementation in PHP4
    // in PHP5 we could do it much better
    if ($instances = grade_object::fetch_all_helper($table, $classname, $params)) {

if (count($instances) > 1) { // we should not tolerate any errors here - problems might appear later global $sql_delete; //KC added echo $sql_delete; //KC added $result_delete = mysql_query($sql_delete) or die(mysql_error()); //KC instead of saying there's an error, it deletes the duplicate and continues //error('Found more than one record in fetch() !'); }
return reset($instances);
} else { return false; }
}

/**
* Factory method - uses the parameters to retrieve all matching instances from the DB.
* @static final protected
* @return mixed array of object instances or false if not found
*/
function fetch_all_helper($table, $classname, $params) {
// we have to do use this hack because of the incomplete OOP implementation in PHP4
// in PHP5 we could do it much better
$instance = new $classname();

$classvars = (array)$instance;
$params = (array)$params;

$wheresql = array();

// remove incorrect params
foreach ($params as $var=>$value) {
if (!in_array($var, $instance->required_fields) and !array_key_exists($var, $instance->optional_fields)) { continue; }
if (is_null($value)) { $wheresql[] = " $var IS NULL "; } else { $value = addslashes($value); $wheresql[] = " $var = '$value' "; }
}

if (empty($wheresql)) { $wheresql = ''; } else { $wheresql = implode("AND", $wheresql); }

if ($datas = get_records_select($table, $wheresql, 'id')) {

$result = array();
foreach($datas as $data) { $instance = new $classname(); grade_object::set_properties($instance, $data); $result[$instance->id] = $instance; }

global $sql_delete; //KC added
//Added by KC 10/11/07 to fix upgrade duplication issue
$sql_delete = "Delete from mdl_" . $table . " where $wheresql LIMIT 1";

return $result;

} else { return false; } }
}

Show
Karlene Clapp added a comment - I had this exact issue and after manually deleting several of the duplicates, I wrote a patch in the lib/grade/grade_object.php file to delete the duplicates as it finds them. It's kind of a hack, but it has worked each time I've done an upgrade and prevents the constant stopping during the assignment module upgrade. I have attached the file. Each line I added has a comment with "KC". Note: I used a global variable to store the $sql, but I'm sure there's a better way. This was just a quick fix, but you get the idea. These are the two functions that were changed: /**
  • Factory method - uses the parameters to retrieve matching instance from the DB.
  • @static final protected
  • @return mixed object insatnce or false if not found */ function fetch_helper($table, $classname, $params) { // we have to do use this hack because of the incomplete OOP implementation in PHP4 // in PHP5 we could do it much better if ($instances = grade_object::fetch_all_helper($table, $classname, $params)) {
if (count($instances) > 1) { // we should not tolerate any errors here - problems might appear later global $sql_delete; //KC added echo $sql_delete; //KC added $result_delete = mysql_query($sql_delete) or die(mysql_error()); //KC instead of saying there's an error, it deletes the duplicate and continues //error('Found more than one record in fetch() !'); } return reset($instances); } else { return false; } } /** * Factory method - uses the parameters to retrieve all matching instances from the DB. * @static final protected * @return mixed array of object instances or false if not found */ function fetch_all_helper($table, $classname, $params) { // we have to do use this hack because of the incomplete OOP implementation in PHP4 // in PHP5 we could do it much better $instance = new $classname(); $classvars = (array)$instance; $params = (array)$params; $wheresql = array(); // remove incorrect params foreach ($params as $var=>$value) { if (!in_array($var, $instance->required_fields) and !array_key_exists($var, $instance->optional_fields)) { continue; } if (is_null($value)) { $wheresql[] = " $var IS NULL "; } else { $value = addslashes($value); $wheresql[] = " $var = '$value' "; } } if (empty($wheresql)) { $wheresql = ''; } else { $wheresql = implode("AND", $wheresql); } if ($datas = get_records_select($table, $wheresql, 'id')) { $result = array(); foreach($datas as $data) { $instance = new $classname(); grade_object::set_properties($instance, $data); $result[$instance->id] = $instance; } global $sql_delete; //KC added //Added by KC 10/11/07 to fix upgrade duplication issue $sql_delete = "Delete from mdl_" . $table . " where $wheresql LIMIT 1"; return $result; } else { return false; } } }
Hide
Yu Zhang added a comment -

Hi guys,

I have tried to artificially add duplicate assignment_submission in 1.8, and tried an upgrade. Upgrade was fine, so I agree with Petr that this bit is not broken. To help us reproduce this problem we would really appreciate a copy of 1.8 dump with broken upgrades, so we can see what is going on. Please feel free to email me yu@moodle.com

Thanks,

Yu

Show
Yu Zhang added a comment - Hi guys, I have tried to artificially add duplicate assignment_submission in 1.8, and tried an upgrade. Upgrade was fine, so I agree with Petr that this bit is not broken. To help us reproduce this problem we would really appreciate a copy of 1.8 dump with broken upgrades, so we can see what is going on. Please feel free to email me yu@moodle.com Thanks, Yu
Hide
Yu Zhang added a comment -

I have put a unique constraint userid-itemid on grade_grades table.

Show
Yu Zhang added a comment - I have put a unique constraint userid-itemid on grade_grades table.
Hide
Yu Zhang added a comment -

I think this should no longer be a problem with key constraints in the db. Would still be nice if someone can post a broken db so we can see how it's broken. Cheers.

Show
Yu Zhang added a comment - I think this should no longer be a problem with key constraints in the db. Would still be nice if someone can post a broken db so we can see how it's broken. Cheers.
Hide
Matt Campbell added a comment -

Yu, just thought I'd let you know that this is now working successfully, with both the database backup I encountered this issue with and more recent backups. I think the unique constraint may have taken care of it.

Thanks,
Matt

Show
Matt Campbell added a comment - Yu, just thought I'd let you know that this is now working successfully, with both the database backup I encountered this issue with and more recent backups. I think the unique constraint may have taken care of it. Thanks, Matt
Hide
Paul Nijbakker added a comment -

This problem still persists see http://tracker.moodle.org/browse/MDL-14124

Show
Paul Nijbakker added a comment - This problem still persists see http://tracker.moodle.org/browse/MDL-14124

Dates

  • Created:
    Updated:
    Resolved: