Moodle

Allowing negative marks for questions

Details

  • Type: New Feature New Feature
  • Status: Closed Closed
  • Priority: Major Major
  • Resolution: Fixed
  • Affects Version/s: 1.4.5
  • Fix Version/s: 2.1
  • Component/s: Quiz
  • Labels:
    None
  • Environment:
    All
  • Affected Branches:
    MOODLE_14_STABLE
  • Fixed Branches:
    MOODLE_21_STABLE

Description

I needed a way to simulate Advanced Placement multiple choice tests in which incorrect answers deduct .25% from the total score of the test.

I made two very minor changes to mod/quiz/lib.php to do this, and it seems to work.

I only tested it on multiple choice (with both one and multiple answers), and Cloze questions (pull-down style only). For my purposes, I made the correct answer be 100%, and all incorrect choices be -25%. This with the changes in lib.php seem to work.

This thread describes my changes, and Martin asked that I report it as a bug.

http://moodle.org/mod/forum/discuss.php?d=9704

Issue Links

Activity

Hide
Martin Dougiamas added a comment -

From Gustav Delius (gwd2 at york.ac.uk) Thursday, 30 December 2004, 07:08 PM:

Hi Jill,

if you still haven't observed any negative side effects of this hack then I would like to put it into Moodle 1.5.

Gustav

From Gustav Delius (gwd2 at york.ac.uk) Saturday, 7 May 2005, 01:39 AM:

Ah, I almost forgot about this. Our new code again rules out negative marks for questions. I am not sure why, it just seemed natural to us. I will see if I find the time to fix that, it shouldn't be too difficult. I am upgrading the priority on this.

From Gustav Delius (gwd2 at york.ac.uk) Thursday, 23 February 2006, 05:49 PM:

This subject has come up again on the forums, see for example http://moodle.org/mod/forum/discuss.php?d=40310&parent=185678

From Gustav Delius (gwd2 at york.ac.uk) Thursday, 23 February 2006, 05:51 PM:

Also see http://moodle.org/mod/forum/discuss.php?d=40128.

We could implement another quiz option to allow negative marks for questions.

From John Isner (john.isner at gmail.com) Tuesday, 11 July 2006, 08:04 PM:

Standardized tests such as the SAT deduct 1/4 point for each wrong answer to a MCQ. It is very difficult to simulate those tests in moodle without this feature.

Show
Martin Dougiamas added a comment - From Gustav Delius (gwd2 at york.ac.uk) Thursday, 30 December 2004, 07:08 PM: Hi Jill, if you still haven't observed any negative side effects of this hack then I would like to put it into Moodle 1.5. Gustav From Gustav Delius (gwd2 at york.ac.uk) Saturday, 7 May 2005, 01:39 AM: Ah, I almost forgot about this. Our new code again rules out negative marks for questions. I am not sure why, it just seemed natural to us. I will see if I find the time to fix that, it shouldn't be too difficult. I am upgrading the priority on this. From Gustav Delius (gwd2 at york.ac.uk) Thursday, 23 February 2006, 05:49 PM: This subject has come up again on the forums, see for example http://moodle.org/mod/forum/discuss.php?d=40310&parent=185678 From Gustav Delius (gwd2 at york.ac.uk) Thursday, 23 February 2006, 05:51 PM: Also see http://moodle.org/mod/forum/discuss.php?d=40128. We could implement another quiz option to allow negative marks for questions. From John Isner (john.isner at gmail.com) Tuesday, 11 July 2006, 08:04 PM: Standardized tests such as the SAT deduct 1/4 point for each wrong answer to a MCQ. It is very difficult to simulate those tests in moodle without this feature.
Hide
Paulo Matos added a comment -

This patch allows multichoice and multianswer questiontypes to have negative marks. I choose this two types because it seemd to me the most logical ones. The total sum of all questions on a quiz is set to zero if negative.

Show
Paulo Matos added a comment - This patch allows multichoice and multianswer questiontypes to have negative marks. I choose this two types because it seemd to me the most logical ones. The total sum of all questions on a quiz is set to zero if negative.
Hide
Tim Hunt added a comment -

Some feedback on the patch:

The reason this bug is non-trivial is that you have to consider what happens in adaptive mode as well as non-adaptive mode. Your patch does not consider that.

Also, "//paulo.matos: ..." comments are not helpful. CVS tracks who changed what, so names in comments just clutter the code. So only use comments to describe what the code is doing, like "// Make sure sumgrades are non-negative."

Comments like "// This should not be here, ..." do not inspire confidence (questionlib line ~1060). In fact, this comment confused me. It was clearer to me what was going on when I ignored the comment and worked out what the code is doing.

I think that any patch that fixes this should treat all question types the same. This is a pain becuase the

Show
Tim Hunt added a comment - Some feedback on the patch: The reason this bug is non-trivial is that you have to consider what happens in adaptive mode as well as non-adaptive mode. Your patch does not consider that. Also, "//paulo.matos: ..." comments are not helpful. CVS tracks who changed what, so names in comments just clutter the code. So only use comments to describe what the code is doing, like "// Make sure sumgrades are non-negative." Comments like "// This should not be here, ..." do not inspire confidence (questionlib line ~1060). In fact, this comment confused me. It was clearer to me what was going on when I ignored the comment and worked out what the code is doing. I think that any patch that fixes this should treat all question types the same. This is a pain becuase the
Hide
Tim Hunt added a comment -

(Dratted thing. I had not finished. Why did it suddenly submit my comment when I hit return, to continue ...)

I think that any patch that fixes this should treat all question types the same. This is a pain because the

$state->raw_grade = min(max((float) $state->raw_grade, 0.0), 1.0)

  • $question->maxgrade;

line of code is copied and pasted into each question type. Probably we need to start by making a method in the base class that applies this logic, and then call that function from each question type. Then we can fix this bug in one place.

So, a good attempt to fix this bug, but it needs more work before it can be committed to core.

Also, I feel that this counts as a new feature, so I would be reluctant to add it to the 1.6 stable branch. I would rather put in into HEAD. By all means you can make a patch available to people who want to patch their own 1.6 installs, but for core, we need a patch against HEAD.

Thanks for your help.

Show
Tim Hunt added a comment - (Dratted thing. I had not finished. Why did it suddenly submit my comment when I hit return, to continue ...) I think that any patch that fixes this should treat all question types the same. This is a pain because the $state->raw_grade = min(max((float) $state->raw_grade, 0.0), 1.0)
  • $question->maxgrade;
line of code is copied and pasted into each question type. Probably we need to start by making a method in the base class that applies this logic, and then call that function from each question type. Then we can fix this bug in one place. So, a good attempt to fix this bug, but it needs more work before it can be committed to core. Also, I feel that this counts as a new feature, so I would be reluctant to add it to the 1.6 stable branch. I would rather put in into HEAD. By all means you can make a patch available to people who want to patch their own 1.6 installs, but for core, we need a patch against HEAD. Thanks for your help.
Hide
Paulo Matos added a comment -

> Also, "//paulo.matos: ..." comments are not helpful.
Sorry for that I'll clean that on my next patches. I use it to track my changes while working on the patch.

> Comments like "// This should not be here, ..." do not inspire confidence (questionlib line ~1060). In fact, this comment confused me. It was clearer to me what was going on when I ignored the comment and worked out what the code is doing.

Ok, I might not expressed quite well on that comment! This comment was about the block code, that is runned every time, even if does not have penalties or timelimit constraints. It should be like a note to the merger. I'll post a new version of the patch with it more clear (hopefully).

Show
Paulo Matos added a comment - > Also, "//paulo.matos: ..." comments are not helpful. Sorry for that I'll clean that on my next patches. I use it to track my changes while working on the patch. > Comments like "// This should not be here, ..." do not inspire confidence (questionlib line ~1060). In fact, this comment confused me. It was clearer to me what was going on when I ignored the comment and worked out what the code is doing. Ok, I might not expressed quite well on that comment! This comment was about the block code, that is runned every time, even if does not have penalties or timelimit constraints. It should be like a note to the merger. I'll post a new version of the patch with it more clear (hopefully).
Hide
Paulo Matos added a comment -

Ok, here's the new version of the patch.

The comments on the previous one were doubts about the block:
// Ensure that the grade does not go down
$state->grade = max($state->grade, $state->last_graded->grade);

I concluded that this block is specific to penalties/timelimit/closing time, and coded accordingly.

Cheers,

Paulo

Show
Paulo Matos added a comment - Ok, here's the new version of the patch. The comments on the previous one were doubts about the block: // Ensure that the grade does not go down $state->grade = max($state->grade, $state->last_graded->grade); I concluded that this block is specific to penalties/timelimit/closing time, and coded accordingly. Cheers, Paulo
Hide
Paulo Matos added a comment -

Tim> I think that any patch that fixes this should treat all question types the same.
I choosed multichoice and multianswer questiontypes to have negative marks because their were the single one's where you can set a negative mark for an answer through the UI.

Show
Paulo Matos added a comment - Tim> I think that any patch that fixes this should treat all question types the same. I choosed multichoice and multianswer questiontypes to have negative marks because their were the single one's where you can set a negative mark for an answer through the UI.
Hide
Paulo Matos added a comment -

Here goes the updated patch for 1.8.x, only multichoice and multianswer questions are treated because those are the only ones that allow setting a negative value through UI.

Show
Paulo Matos added a comment - Here goes the updated patch for 1.8.x, only multichoice and multianswer questions are treated because those are the only ones that allow setting a negative value through UI.
Hide
Paulo Matos added a comment -

1.8.6+ version

Show
Paulo Matos added a comment - 1.8.6+ version
Hide
Jitendra Patel added a comment -

I am also facing the Problem but i dont know how to install patch in my site

if anybody can help me

Show
Jitendra Patel added a comment - I am also facing the Problem but i dont know how to install patch in my site if anybody can help me
Hide
Tim Hunt added a comment -

Note: a proper solution to this issue is part of this proposal: http://docs.moodle.org/en/Development:Changing_the_Moodle_Question_Engine (There is currently no schedule for implementing it.)

Show
Tim Hunt added a comment - Note: a proper solution to this issue is part of this proposal: http://docs.moodle.org/en/Development:Changing_the_Moodle_Question_Engine (There is currently no schedule for implementing it.)
Hide
Amanda Doughty added a comment -

Tim> I have been testing various different patches and have found that Vikram's solution: http://moodle.org/mod/forum/discuss.php?d=46862#p501237 mostly works for 1.9.8.

I did take into account your comments in http://tracker.moodle.org/browse/MDL-5324 regarding:

1. Multiple-response multiple-choice questions.
2. When the quiz allows multiple attempts at each question using the penalty-factor system.

Using this patch for questions using the penalty-factor system as well as adaptive mode, I found that unchanged submitted answers (using the individual submit button) were regraded and therefore penalised again when 'submit all and finish' is clicked.

I have therefore added to Vikram's patch to resolve this and initial tests look good. We will carry out further testing, but as I am a relative newbie to Moodle development, I wondered if you would have a quick look and let us know if you spot any immediate flaws. It is a very small patch. I will attach it as negativemarking.diff

Thanks

Show
Amanda Doughty added a comment - Tim> I have been testing various different patches and have found that Vikram's solution: http://moodle.org/mod/forum/discuss.php?d=46862#p501237 mostly works for 1.9.8. I did take into account your comments in http://tracker.moodle.org/browse/MDL-5324 regarding: 1. Multiple-response multiple-choice questions. 2. When the quiz allows multiple attempts at each question using the penalty-factor system. Using this patch for questions using the penalty-factor system as well as adaptive mode, I found that unchanged submitted answers (using the individual submit button) were regraded and therefore penalised again when 'submit all and finish' is clicked. I have therefore added to Vikram's patch to resolve this and initial tests look good. We will carry out further testing, but as I am a relative newbie to Moodle development, I wondered if you would have a quick look and let us know if you spot any immediate flaws. It is a very small patch. I will attach it as negativemarking.diff Thanks
Hide
Tim Hunt added a comment -

Any change that touches the function question_process_responses in lib/questionlib.php is very dangerous. I know from painful experience that change here need a lot of testing to ensure that the do not introduce a subtle bug. I cannot, from a quick look at your patch, say with any certainty whether they are correct or not. They look plausible, and if you have tested and they work in all situations that you care about, then it should be safe for you to use them.

However, the proper solution to this problem is some work I am currently doing that is destined to go into Moodle 2.1. It is described on http://docs.moodle.org/en/Development:Question_Engine_2, and the code which is mostly working, is available from http://github.com/timhunt/Moodle-Question-Engine-2. Since this work is underway, I am not going to do anything about this issue in Moodle 1.9 or 2.0.

Show
Tim Hunt added a comment - Any change that touches the function question_process_responses in lib/questionlib.php is very dangerous. I know from painful experience that change here need a lot of testing to ensure that the do not introduce a subtle bug. I cannot, from a quick look at your patch, say with any certainty whether they are correct or not. They look plausible, and if you have tested and they work in all situations that you care about, then it should be safe for you to use them. However, the proper solution to this problem is some work I am currently doing that is destined to go into Moodle 2.1. It is described on http://docs.moodle.org/en/Development:Question_Engine_2, and the code which is mostly working, is available from http://github.com/timhunt/Moodle-Question-Engine-2. Since this work is underway, I am not going to do anything about this issue in Moodle 1.9 or 2.0.
Hide
Amanda Doughty added a comment -

Ok Tim. Thanks for the advice.

Show
Amanda Doughty added a comment - Ok Tim. Thanks for the advice.
Hide
Tim Hunt added a comment -

Fixed in 2.1 dev now that MDL-20636 has been integrated.

That only too 6.5 years

Show
Tim Hunt added a comment - Fixed in 2.1 dev now that MDL-20636 has been integrated. That only too 6.5 years

People

Vote (28)
Watch (19)

Dates

  • Created:
    Updated:
    Resolved: