Moodle
  1. Moodle
  2. MDL-33477

Editing a marking guide after grading has unexpected behaviours

    Details

    • Rank:
      41376

      Description

      Follow the steps from MDLQA-2489 to reproduce.

      Fails on step 5. An error message appears:

      "You are about to save changes to a marking guide that has already been used for grading. Please indicate if existing grades need to be reviewed. If you set this then the marking guide will be hidden from students until their item is regraded. <select>Mark for regrade / Do not mark for regrade</select>"

      Fails on step 16. Everything is hidden in mod/assign/view.php, but new criterias are visible in "Submissions grading" from the navigation menu right under the current assignment, is it expected?

      Fails on step 29. The warning message does not match the one expected, plus there is no options. This is the message:

      "You are about to save significant changes to a marking guide that has already been used for grading. The gradebook value will be unchanged, but the marking guide will be hidden from students until their item is regraded."

        Activity

        Hide
        Dan Marsden added a comment -

        step 5 is expected behaviour modelled on how the existing Rubrics advanced grading type works - changing the name or description of the marking guide will trigger the regrade warning. If we want to change this it should be logged as an improvement and happen in both Rubric and Marking guide to be consistent.

        It is possible to modify the freq used comments without triggering the warning. so if you remove steps 2/3 in the QA test it should pass that part

        Show
        Dan Marsden added a comment - step 5 is expected behaviour modelled on how the existing Rubrics advanced grading type works - changing the name or description of the marking guide will trigger the regrade warning. If we want to change this it should be logged as an improvement and happen in both Rubric and Marking guide to be consistent. It is possible to modify the freq used comments without triggering the warning. so if you remove steps 2/3 in the QA test it should pass that part
        Hide
        Dan Marsden added a comment -

        step 16 is expected behaviour - the option "Show guide definition to students" defines whether the students are able to see the criteria used by the advanced grading method (same for Rubrics grading type)

        Show
        Dan Marsden added a comment - step 16 is expected behaviour - the option "Show guide definition to students" defines whether the students are able to see the criteria used by the advanced grading method (same for Rubrics grading type)
        Hide
        Dan Marsden added a comment -

        step 29 is also expected behaviour - if any criteria are added it will trigger the force regrade option and not give the user a choice - there's no way the existing code can check to see if a "newly added" criteria is a replacement of a previously deleted criteria. If we would like this to be added we should add a new bug in the tracker for it as a new improvement.

        Show
        Dan Marsden added a comment - step 29 is also expected behaviour - if any criteria are added it will trigger the force regrade option and not give the user a choice - there's no way the existing code can check to see if a "newly added" criteria is a replacement of a previously deleted criteria. If we would like this to be added we should add a new bug in the tracker for it as a new improvement.
        Hide
        Dan Marsden added a comment -

        not quite sure how to flag this issue - the issues raised are either expected behavior or incorrect QA test instructions - I guess we need to update the QA test and then we can close this one? - any improvements can be added as new tracker issues if required.

        Show
        Dan Marsden added a comment - not quite sure how to flag this issue - the issues raised are either expected behavior or incorrect QA test instructions - I guess we need to update the QA test and then we can close this one? - any improvements can be added as new tracker issues if required.
        Hide
        Sam Hemelryk added a comment -

        Hi Eva,

        I've just been speaking to Tim and he mentioned that you were the author of those tests. I've added you as a watcher here to get your thoughts.
        I've read through Fred's comments, and Dan's replies, and I agreed that by the looks of it in all circumstances it is the expected behaviour.
        Looking at the testing instructions I think there are probably a couple of things that could be clarified there.
        Would you mind taking a look at this issue, and at the test instructions and see about improving them.
        I know Tim has asked Fred to do the same.

        Cheers
        Sam

        Show
        Sam Hemelryk added a comment - Hi Eva, I've just been speaking to Tim and he mentioned that you were the author of those tests. I've added you as a watcher here to get your thoughts. I've read through Fred's comments, and Dan's replies, and I agreed that by the looks of it in all circumstances it is the expected behaviour. Looking at the testing instructions I think there are probably a couple of things that could be clarified there. Would you mind taking a look at this issue, and at the test instructions and see about improving them. I know Tim has asked Fred to do the same. Cheers Sam
        Hide
        Frédéric Massart added a comment -

        I agree with you Sam, it looks like it is not an issue as the marking guide works fine even if it does not follow exactly the testing instructions.

        Show
        Frédéric Massart added a comment - I agree with you Sam, it looks like it is not an issue as the marking guide works fine even if it does not follow exactly the testing instructions.
        Hide
        Eva Heinrich added a comment -

        Hi Sam,

        I agree with you and the others that the behaviour around the marking guide is ok and that the testing instructions need to be changed.
        The only change to the implementation I am suggesting is around wording. The options presented to the user are: ‘Mark for regrade’, ‘Do not mark for regrade’. Can this be changed to maybe ‘Identify for regrade’, ‘Do not identify for regrade’ or 'Enforce regrade', 'Do not enforce regrade' as it could be confusing to use 'mark' in a marking context?
        (I am working on re-writing the testing instructions.)

        Regards, Eva

        Show
        Eva Heinrich added a comment - Hi Sam, I agree with you and the others that the behaviour around the marking guide is ok and that the testing instructions need to be changed. The only change to the implementation I am suggesting is around wording. The options presented to the user are: ‘Mark for regrade’, ‘Do not mark for regrade’. Can this be changed to maybe ‘Identify for regrade’, ‘Do not identify for regrade’ or 'Enforce regrade', 'Do not enforce regrade' as it could be confusing to use 'mark' in a marking context? (I am working on re-writing the testing instructions.) Regards, Eva
        Hide
        Eva Heinrich added a comment -

        Can someone check these modified instructions please?

        Test Pre-requisites:
        ▪ assignments created should be of the new assignment module type
        ▪ an existing assignment set to use a marking guide with a complete guide defined, and some marking completed
        ▪ unless otherwise specified testing should be performed as a teacher user
        Test Steps:
        1. Open the marking guide for editing
        2. Change the order and text of the frequently used comments on the marking guide
        3. Save the marking guide
        4. View existing marking records and check that nothing has changed
        5. Open the marking guide for editing
        6. Change the name of the marking guide
        7. Change the description of the marking guide
        8. Change the name of a criterion
        9. Change the descriptions for students of a criterion
        10. Change the descriptions for markers of a criterion
        11. Change the maximum mark allowed for a criterion, making sure that the total marks across all criteria still add up to the maximum mark defined for the assignment
        12. Save the marking guide
        13. Check that a warning has appeared telling you that this marking guide has already been used for marking and that you are presented with two options (‘Mark for regrade’, ‘Do not mark for regrade’)
        14. Select the option ‘Mark for regrade’
        15. Login as a student whose submission had been marked
        16. Check that the student cannot see their individual comments and marks related to the marking guide, but that the unchanged grade remains visible
        17. Log back in as the teacher and regrade the student's submission
        18. Login as the student whose submission has just been marked
        19. Check that the student can now see their individual comments and marks related to the marking guide
        20. Repeat steps 7 through 13
        21. Select the option ‘Do not mark for regrade’
        22. Login as the student who has had a submission marked already
        23. Check that the student can see their individual comments and marks related to the marking guide
        24. Log back in as the teacher
        25. Open the marking guide for editing
        26. Remove one of the criteria
        27. Create a new criterion with the same maximum grade
        28. Save the marking guide (a warning is presented telling you that this marking guide has already been used for marking and that the students’ marking details associated with the marking guide will not be shown to students until remarking)
        29. Login as a student whose submission had been marked
        30. Check that the student cannot see their individual comments and marks related to the marking guide, but that the unchanged grade remains visible
        31. Log back in as the teacher
        32. Return to marking a student who has already been marked
        33. Remark the student
        34. Login as the student who has just had their submission re-marked
        35. Check that the student can now see their individual comments and marks related to the marking guide with the updated information and that the grade is correct

        Show
        Eva Heinrich added a comment - Can someone check these modified instructions please? Test Pre-requisites: ▪ assignments created should be of the new assignment module type ▪ an existing assignment set to use a marking guide with a complete guide defined, and some marking completed ▪ unless otherwise specified testing should be performed as a teacher user Test Steps: 1. Open the marking guide for editing 2. Change the order and text of the frequently used comments on the marking guide 3. Save the marking guide 4. View existing marking records and check that nothing has changed 5. Open the marking guide for editing 6. Change the name of the marking guide 7. Change the description of the marking guide 8. Change the name of a criterion 9. Change the descriptions for students of a criterion 10. Change the descriptions for markers of a criterion 11. Change the maximum mark allowed for a criterion, making sure that the total marks across all criteria still add up to the maximum mark defined for the assignment 12. Save the marking guide 13. Check that a warning has appeared telling you that this marking guide has already been used for marking and that you are presented with two options (‘Mark for regrade’, ‘Do not mark for regrade’) 14. Select the option ‘Mark for regrade’ 15. Login as a student whose submission had been marked 16. Check that the student cannot see their individual comments and marks related to the marking guide, but that the unchanged grade remains visible 17. Log back in as the teacher and regrade the student's submission 18. Login as the student whose submission has just been marked 19. Check that the student can now see their individual comments and marks related to the marking guide 20. Repeat steps 7 through 13 21. Select the option ‘Do not mark for regrade’ 22. Login as the student who has had a submission marked already 23. Check that the student can see their individual comments and marks related to the marking guide 24. Log back in as the teacher 25. Open the marking guide for editing 26. Remove one of the criteria 27. Create a new criterion with the same maximum grade 28. Save the marking guide (a warning is presented telling you that this marking guide has already been used for marking and that the students’ marking details associated with the marking guide will not be shown to students until remarking) 29. Login as a student whose submission had been marked 30. Check that the student cannot see their individual comments and marks related to the marking guide, but that the unchanged grade remains visible 31. Log back in as the teacher 32. Return to marking a student who has already been marked 33. Remark the student 34. Login as the student who has just had their submission re-marked 35. Check that the student can now see their individual comments and marks related to the marking guide with the updated information and that the grade is correct
        Hide
        Tim Barker added a comment -

        I feel that to clarify the test further, the following modifications are required:

        1. The pre-requisites must include a step that specifies frequently used comments must be included. Step #2 requires this data yet no-where are we asked to set it up and as it is not mandatory to include a frequently used comment, the creator of the marking guide is no compulsion to add one.
        2. Step 4: "View existing marking records and check that nothing has changed" I think that you mean: "check the existing student's grading records (i.e. the grading form) for this assignment and check that they have not changed." and they had changed! The order of the Frequently used comments and the text that they contain had changed. 1) I'm not really sure that I'm checking in the correct place and 2) I'm not entirely sure what is supposed to have changed. If nothing is supposed to have changed then this test fails at step 4!
        3. "7. Change the description of the marking guide". The description field is incorrectly labelled "Description for students". This is a bug! I'm assuming that the correct place for the "description for students" field label is with the criterion.
        4. Step 14. A click on the continue button is required after selecting "Mark for regrade". Same goes for step 21, 28

        I am of the opinion, and the issues we have had with this test are testament to, the fact that long tests are difficult to follow and time-consuming to run. I also feel that there are 3 distinct test scenarios here.

        • Edit a marking guide and Mark for regrade.
        • Edit a marking guide and Do not mark for regrade.
        • Edit a marking guide and replace an existing criterion

        Breaking down the tests in this way make the tests easier to follow, faster to run and give greater granularity to the test results - therefore assisting with defect reporting.

        I can make the modifications to the tests myself but I will need clarification on point 2 above. Will someone please clarify this for me. In the meantime I will raise a non-blocking issue with the "Description" field label and create three new tests based upon this one.

        Show
        Tim Barker added a comment - I feel that to clarify the test further, the following modifications are required: The pre-requisites must include a step that specifies frequently used comments must be included. Step #2 requires this data yet no-where are we asked to set it up and as it is not mandatory to include a frequently used comment, the creator of the marking guide is no compulsion to add one. Step 4: "View existing marking records and check that nothing has changed" I think that you mean: "check the existing student's grading records (i.e. the grading form) for this assignment and check that they have not changed." and they had changed! The order of the Frequently used comments and the text that they contain had changed. 1) I'm not really sure that I'm checking in the correct place and 2) I'm not entirely sure what is supposed to have changed. If nothing is supposed to have changed then this test fails at step 4! "7. Change the description of the marking guide". The description field is incorrectly labelled "Description for students". This is a bug! I'm assuming that the correct place for the "description for students" field label is with the criterion. Step 14. A click on the continue button is required after selecting "Mark for regrade". Same goes for step 21, 28 I am of the opinion, and the issues we have had with this test are testament to, the fact that long tests are difficult to follow and time-consuming to run. I also feel that there are 3 distinct test scenarios here. Edit a marking guide and Mark for regrade. Edit a marking guide and Do not mark for regrade. Edit a marking guide and replace an existing criterion Breaking down the tests in this way make the tests easier to follow, faster to run and give greater granularity to the test results - therefore assisting with defect reporting. I can make the modifications to the tests myself but I will need clarification on point 2 above. Will someone please clarify this for me. In the meantime I will raise a non-blocking issue with the "Description" field label and create three new tests based upon this one.
        Hide
        Eva Heinrich added a comment -

        Hi Tim,
        Breaking the test up into smaller parts sounds like a good idea.
        Response to your point 2: I had missed out fixing step 4. The following should be correct:

        1. Open the marking guide for editing
        2. Change the order and text of the frequently used comments on the marking guide
        3. Save the marking guide (the marking guide saves, no warning message about a potential need for a regrade is triggered)
        4. Login as the student who has had a submission marked already
        5. Check that the student can see their individual comments and marks related to the marking guide and their unchanged grade

        What I am focusing on in these tests is the individual feedback for a student linked to the marking guide (the comments to a criterion and the marks per criterion). The variations in the tests are if the student can see these details with or without regrade, and in the later test (where the criteria have changed), if the student sees the correct version of the criteria. I am also checking that students see the correct grade information.
        I am not looking at the changes to the marking guide itself (as visible to students in http://qa.moodle.net/grade/grading/form/guide).

        Your point about the 'Description to students field': The idea is to have (at some stage) a parallel field called 'Description for Markers' (I am not sure why/where this has fallen of the specification; maybe I missed it). The purpose of these two fields would be to give information relevant to the marking of the assignment overall. An example for a description to markers could be 'Provide an overall comment to each student pointing out one strong point and one area for improvement'. This description for markers (like the ones for the criteria) are not shown to students; these descriptions are there for the communication within the marking team.
        As we do not have the 'Description for Markers' field at the moment, you could get the name changed to just 'Description'.

        Regards, Eva

        Show
        Eva Heinrich added a comment - Hi Tim, Breaking the test up into smaller parts sounds like a good idea. Response to your point 2: I had missed out fixing step 4. The following should be correct: 1. Open the marking guide for editing 2. Change the order and text of the frequently used comments on the marking guide 3. Save the marking guide (the marking guide saves, no warning message about a potential need for a regrade is triggered) 4. Login as the student who has had a submission marked already 5. Check that the student can see their individual comments and marks related to the marking guide and their unchanged grade What I am focusing on in these tests is the individual feedback for a student linked to the marking guide (the comments to a criterion and the marks per criterion). The variations in the tests are if the student can see these details with or without regrade, and in the later test (where the criteria have changed), if the student sees the correct version of the criteria. I am also checking that students see the correct grade information. I am not looking at the changes to the marking guide itself (as visible to students in http://qa.moodle.net/grade/grading/form/guide ). Your point about the 'Description to students field': The idea is to have (at some stage) a parallel field called 'Description for Markers' (I am not sure why/where this has fallen of the specification; maybe I missed it). The purpose of these two fields would be to give information relevant to the marking of the assignment overall. An example for a description to markers could be 'Provide an overall comment to each student pointing out one strong point and one area for improvement'. This description for markers (like the ones for the criteria) are not shown to students; these descriptions are there for the communication within the marking team. As we do not have the 'Description for Markers' field at the moment, you could get the name changed to just 'Description'. Regards, Eva
        Hide
        Dan Marsden added a comment -

        whoops - that's a bug with the lang - will push a fix for that on this tracker issue in the next hour or so..

        Eva - I talked with Charles about the 2 description fields in the spec - the existing description field that is shown comes from the core grading_definitions table which isn't part of the plugin itself - to add a 2nd description field may require changes to the core grading_definitions table which would have made this harder to get into Moodle core as a plugin.

        Now the plugin is in core - we can look at some of those other features (if required) and try to get them into Moodle core as seperate improvements.

        Show
        Dan Marsden added a comment - whoops - that's a bug with the lang - will push a fix for that on this tracker issue in the next hour or so.. Eva - I talked with Charles about the 2 description fields in the spec - the existing description field that is shown comes from the core grading_definitions table which isn't part of the plugin itself - to add a 2nd description field may require changes to the core grading_definitions table which would have made this harder to get into Moodle core as a plugin. Now the plugin is in core - we can look at some of those other features (if required) and try to get them into Moodle core as seperate improvements.
        Hide
        Eva Heinrich added a comment -

        Hi Dan,

        I am a bit lost now - what bug are you referring to?
        Thanks for the explanation on the second description field - I suggest we hold back on that for the moment.

        Regards, Eva

        Show
        Eva Heinrich added a comment - Hi Dan, I am a bit lost now - what bug are you referring to? Thanks for the explanation on the second description field - I suggest we hold back on that for the moment. Regards, Eva
        Hide
        Dan Marsden added a comment -

        no worries - when editing an existing marking guide the text "description for students" is displayed beside the main description field - it should just say "description"

        Show
        Dan Marsden added a comment - no worries - when editing an existing marking guide the text "description for students" is displayed beside the main description field - it should just say "description"
        Hide
        Dan Marsden added a comment -

        bouncing this lang change up for integration - the other items on this relate to improving test details which can be done separately

        Show
        Dan Marsden added a comment - bouncing this lang change up for integration - the other items on this relate to improving test details which can be done separately
        Hide
        Dan Poltawski added a comment -

        Looks good, passing thanks.

        Show
        Dan Poltawski added a comment - Looks good, passing thanks.
        Hide
        Dan Poltawski added a comment -

        (discussed with Fredd and removing QA label as this is no longer QA blocking, but also not resetting qa test)

        Show
        Dan Poltawski added a comment - (discussed with Fredd and removing QA label as this is no longer QA blocking, but also not resetting qa test)
        Hide
        Dan Marsden added a comment -

        cool - thanks Dan!

        Show
        Dan Marsden added a comment - cool - thanks Dan!
        Hide
        Eloy Lafuente (stronk7) added a comment -

        Big thanks for the effort. This is now part of Moodle upstream. Let's wait for regressions, yay! LOL

        Ciao

        Show
        Eloy Lafuente (stronk7) added a comment - Big thanks for the effort. This is now part of Moodle upstream. Let's wait for regressions, yay! LOL Ciao

          People

          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: