Course Content Duplicate error when grade letters changed

Hi, Jose.

Thanks for reporting that. I couldn't find a report of this already in Tracker.

Perhaps the grade value duplication is not necessary.

Feel free to help us work on this issue. If you are able to provide a patch, please add a patch label so we will spot it.

Hi-
I haven't duplicated any courses however when I try to change the default letters using Settings>Site administration>Grades>Letters, I get a similar error. I don't know if stack trace info is necessary so I've added that as well. Moodle 2.2

Debug info: Duplicate entry '1-80.00000-B-' for key 'mdl_gradlett_conlowlet_uix'
UPDATE mdl_grade_letters SET letter = ?,lowerboundary = ?,contextid = ? WHERE id=?
[array (
0 => 'B-',
1 => 80,
2 => '1',
3 => 1,
)]

Stack trace:

line 397 of /lib/dml/moodle_database.php: dml_write_exception thrown
line 985 of /lib/dml/mysqli_native_moodle_database.php: call to moodle_database->query_end()
line 1017 of /lib/dml/mysqli_native_moodle_database.php: call to mysqli_native_moodle_database->update_record_raw()
line 168 of /grade/edit/letter/index.php: call to mysqli_native_moodle_database->update_record()

Please disregard this comment.

The same problem also occurs in version 2.0 during restore the course into the same course with merging data option.
Instructions:
1. Create course with some content.
2. Go Course Administration, Grades, Settings, Letters, Edit Grade Letters & change any of them.
3. Backup course.
4. Restore the course into the same course with merging data option.

We've seen some upgrades from Moodle 1.9 to Moodle 2.0 and 2.1 fail because the mdl_grade_letters table has duplicate values in it. This problem doesn't seem to be just limited to Moodle 2.x.

See MDL-30515 for more info on the upgrade problems.

I can duplicate this problem as well myself. This only happens when the activity being duplicated has user grades associated with it. I tried this with an assignment that had no submissions and the problem didn't occur. When I added a grade value for a single user to the assignment, the error happened duplicate the activity.

Here is a more detail debugging message for the problem:

• Debug info: Duplicate entry '57-0.00000-NC' for key 'mdl_gradlett_conlowlet_uix'
  INSERT INTO mdl_grade_letters (lowerboundary,letter,contextid) VALUES(?,?,?)
  [array (
  0 => '0.00000',
  1 => 'NC',
  2 => '57',
  )]
  Stack trace:
  line 397 of /lib/dml/moodle_database.php: dml_write_exception thrown
  line 893 of /lib/dml/mysqli_native_moodle_database.php: call to moodle_database->query_end()
  line 935 of /lib/dml/mysqli_native_moodle_database.php: call to mysqli_native_moodle_database->insert_record_raw()
  line 268 of /backup/moodle2/restore_stepslib.php: call to mysqli_native_moodle_database->insert_record()
  line 131 of /backup/util/plan/restore_structure_step.class.php: call to restore_gradebook_structure_step->process_grade_letter()
  line 103 of /backup/util/helper/restore_structure_parser_processor.class.php: call to restore_structure_step->process()
  line 125 of /backup/util/xml/parser/processors/grouped_parser_processor.class.php: call to restore_structure_parser_processor->dispatch_chunk()
  line 91 of /backup/util/helper/restore_structure_parser_processor.class.php: call to grouped_parser_processor->postprocess_chunk()
  line 148 of /backup/util/xml/parser/processors/simplified_parser_processor.class.php: call to restore_structure_parser_processor->postprocess_chunk()
  line 92 of /backup/util/xml/parser/processors/progressive_parser_processor.class.php: call to simplified_parser_processor->process_chunk()
  line 169 of /backup/util/xml/parser/progressive_parser.class.php: call to progressive_parser_processor->receive_chunk()
  line 253 of /backup/util/xml/parser/progressive_parser.class.php: call to progressive_parser->publish()
  line ? of unknownfile: call to progressive_parser->end_tag()
  line 158 of /backup/util/xml/parser/progressive_parser.class.php: call to xml_parse()
  line 137 of /backup/util/xml/parser/progressive_parser.class.php: call to progressive_parser->parse()
  line 105 of /backup/util/plan/restore_structure_step.class.php: call to progressive_parser->process()
  line 153 of /backup/util/plan/base_task.class.php: call to restore_structure_step->execute()
  line 148 of /backup/util/plan/base_plan.class.php: call to base_task->execute()
  line 157 of /backup/util/plan/restore_plan.class.php: call to base_plan->execute()
  line 302 of /backup/controller/restore_controller.class.php: call to restore_plan->execute()
  line 90 of /course/modduplicate.php: call to restore_controller->execute_plan()
  

I just came up with a patch to fix this problem and it's available in my Github account:

• https://github.com/jfilip/moodle/compare/MOODLE_21_STABLE...MDL-29598_m21
• https://github.com/jfilip/moodle/tree/MDL-29598_m21

I'm not sure if there is a place higher up in the restore process that this needs to be fixed but the patch I have is the most obvious solution to the problem.

We're running 2.2.2 and have reproduced this error. I can't believe we haven't come across it sooner. I guess not many have used the duplicate feature yet.

Shouldn't this be marked critical?

I received the following error after completing Step 2:

Debug info: Duplicate entry '85-83.00000-B' for key 'mdl_gradlett_conlowlet_uix'
UPDATE mdl_grade_letters SET letter = ?,lowerboundary = ?,contextid = ? WHERE id=?
[array (
0 => 'B',
1 => 83,
2 => '85',
3 => 573,
)]
Stack trace:

line 397 of /lib/dml/moodle_database.php: dml_write_exception thrown
line 999 of /lib/dml/mysqli_native_moodle_database.php: call to moodle_database->query_end()
line 1031 of /lib/dml/mysqli_native_moodle_database.php: call to mysqli_native_moodle_database->update_record_raw()
line 168 of /grade/edit/letter/index.php: call to mysqli_native_moodle_database->update_record()

Thanks for the report and patch Jose and Justin. I assigned it to myself to continue the work of http://tracker.moodle.org/browse/MDL-30515 in the next sprint

I've replicated the problem restoring a course backup with the "Restore into this course" and "Merge the backup course into this course" options, this patch should also get rid of the duplicate problem, which I failed to reproduce. Submitting for peer review.

Thanks Jose, David. The patch looks good to me. Push whenever ready!

The main moodle.git repository has just been updated with latest weekly modifications. You may wish to rebase your PULL branches to simplify history and avoid any possible merge conflicts. This would also make integrator's life easier next week.

TIA and ciao

HI Eloy, rebased

Integrated, thanks! (22, 23 & master)

We are running 2.3.1 (12.07.26) and are not in a position to run a full upgrade right now but would really like to implement this particular fix as it's causing major issues with our current course content. Is this something we could patch independently of running a full upgrade? Thanks in advance!!

Not sure if it is related but am getting the following error in 2.2 during restore (merging in current course)

INSERT INTO mdl_grade_grades (userid,rawgrade,rawgrademax,rawgrademin,rawscaleid,usermodified,finalgrade,hidden,locked,locktime,exported,overridden,excluded,feedback,feedbackformat,information,informationformat,timecreated,timemodified,itemid) VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)
[array (
0 => '2',
1 => NULL,
2 => '100.00000',
3 => '0.00000',
4 => NULL,
5 => NULL,
6 => '0.00000',
7 => '0',
8 => '0',
9 => '0',
10 => '0',
11 => '0',
12 => '0',
13 => NULL,
14 => '0',
15 => NULL,
16 => '0',
17 => NULL,
18 => NULL,
19 => '1',
)]
Stack trace:
line 397 of /lib/dml/moodle_database.php: dml_write_exception thrown
line 973 of /lib/dml/mysqli_native_moodle_database.php: call to moodle_database->query_end()
line 1015 of /lib/dml/mysqli_native_moodle_database.php: call to mysqli_native_moodle_database->insert_record_raw()
line 222 of /backup/moodle2/restore_stepslib.php: call to mysqli_native_moodle_database->insert_record()
line 131 of /backup/util/plan/restore_structure_step.class.php: call to restore_gradebook_structure_step->process_grade_grade()
line 103 of /backup/util/helper/restore_structure_parser_processor.class.php: call to restore_structure_step->process()
line 125 of /backup/util/xml/parser/processors/grouped_parser_processor.class.php: call to restore_structure_parser_processor->dispatch_chunk()
line 91 of /backup/util/helper/restore_structure_parser_processor.class.php: call to grouped_parser_processor->postprocess_chunk()
line 148 of /backup/util/xml/parser/processors/simplified_parser_processor.class.php: call to restore_structure_parser_processor->postprocess_chunk()
line 92 of /backup/util/xml/parser/processors/progressive_parser_processor.class.php: call to simplified_parser_processor->process_chunk()
line 169 of /backup/util/xml/parser/progressive_parser.class.php: call to progressive_parser_processor->receive_chunk()
line 253 of /backup/util/xml/parser/progressive_parser.class.php: call to progressive_parser->publish()
line ? of unknownfile: call to progressive_parser->end_tag()
line 158 of /backup/util/xml/parser/progressive_parser.class.php: call to xml_parse()
line 137 of /backup/util/xml/parser/progressive_parser.class.php: call to progressive_parser->parse()
line 105 of /backup/util/plan/restore_structure_step.class.php: call to progressive_parser->process()
line 153 of /backup/util/plan/base_task.class.php: call to restore_structure_step->execute()
line 148 of /backup/util/plan/base_plan.class.php: call to base_task->execute()
line 157 of /backup/util/plan/restore_plan.class.php: call to base_plan->execute()
line 315 of /backup/controller/restore_controller.class.php: call to restore_plan->execute()
line 147 of /backup/util/ui/restore_ui.class.php: call to restore_controller->execute_plan()
line 46 of /backup/restore.php: call to restore_ui->execute()

Will try to debug it tomorrow. Everything went fine in master.
Thanks

Hi Ankit,

It doesn't seems to be related with this issue, the error your are receiving is related with grade_grades and the grade letters are only representations of grades, they don't have a DB association with grade_grades.

I'm unable to reproduce the problem, is possible that you have corrupted grade records in your DB?

Ankit, I've reproduced the problem in 2.2 (integration) in a course with some activities and without system grade letters overwritten, but I can't reproduced in the latest 2.2 weekly release. I'll continue finding the origin of the problem when I finish the meeting

Hi Ankit, I've also been able to reproduce the problem in 22 STABLE, you only need to have a course with students and a graded activity, when you restore the course backup merging changes the expection is thrown

Adding a link to http://tracker.moodle.org/browse/MDL-32644

Hi Susan,
you can possibly pick these commits from our integration.git repository or even better, cherry-pick it soon (by this week) from our upstream moodle.git repository:

integration:
http://git.moodle.org/gw?p=integration.git&a=search&h=refs%2Fheads%2FMOODLE_23_STABLE&st=commit&s=MDL-29598
moodle (will show up by soon after weekly roll):
http://git.moodle.org/gw?p=moodle.git&a=search&h=refs%2Fheads%2FMOODLE_23_STABLE&st=commit&s=MDL-29598

The patch doesn't seem to affect any upgrade data (or path) as it puts in more checks before inserting records. However i would recommend here backups and extra testing.

As always, best approach is still working towards upgrading moodle itself.

ps: i'm curious as to what would be blocking a 2.3.1 --> upcoming 2.3.2 though.

Passing the test as it is working on other branches. I was not able to test it in 22 because of the linked issue.
Thanks

I'm so proud...of you, many thanks!

http://youtu.be/n64CdfDRnZY

Closing as fixed, ciao