Moodle

Multichoice question with multianswer and custom score: illogic questionsum and behaviour

Details

  • Type: Bug Bug
  • Status: Closed Closed
  • Priority: Minor Minor
  • Resolution: Fixed
  • Affects Version/s: 1.8
  • Fix Version/s: 1.8.4, 1.9
  • Component/s: Lesson
  • Labels:
    None
  • Environment:
    Standard on Linux
  • Database:
    MySQL
  • Affected Branches:
    MOODLE_18_STABLE
  • Fixed Branches:
    MOODLE_18_STABLE, MOODLE_19_STABLE

Description

If you have a custom question (multichoice) with more than one answer that is right, ie that has more than one answer with a score > 0
example:

  • how many nails do mammals have on one foot?
  • 1 (15)
  • 2 (15)
  • 3 (0)
  • 4 (15)
  • 5 (15)
  • first weird behaviour:
  • the total of that question is 15 but should be 60 (the answer is correct if you choos all of the items!)
  • let select the first 2 answers
  • the feedback is set to "wrong answer"
  • you get 15 points out of 15

============= FIRST ISSUE ================
I understand the need of scaling the answers from one totally correct, down to totally wrong ones, as for

  • Do mammals lay eggs?
  • never (90)
  • some peculiar species (100)
  • yes (0)
    in which the right answer is the second but the first is almost right...
    This small sub-issue is resolved assigning a maxvalue to the question itself, and the score given for that question is min(question_max_value, useranswer_total_score) or just using multichoice-one answer only question type!

Comming back to the main first issue, it is resolved by changing the lines in /mod/lesson/locallib.php near line 1474
// Find the highest possible score per page to get our total
foreach ($answers as $answer) {
if(!isset($bestscores[$answer->pageid])) { $bestscores[$answer->pageid] = $answer->score; } else if ($bestscores[$answer->pageid] < $answer->score) { $bestscores[$answer->pageid] = $answer->score; } }
}

in

// Find the highest possible score per page to get our total
foreach ($answers as $answer) {
if(!isset($bestscores[$answer->pageid])) { $bestscores[$answer->pageid] = 0; }
$bestscores[$answer->pageid] += $answer->score);
}

=========== SECOND ISSUE =========
The second issue should be resolved in file /mod/lesson/action/continue.php rewriting the logic of how the correctness is evaluated: in the example case, you got half the question right and no wrong answers!
It should be implemented as a tristate variable.

I did implement in another way:

  • first set to false a flag for wrong answers ($wronguseranswer)!
  • add a flag for incemplete answers that defaults to false ($missinganswers)
  • in the check cycle set it to true if the user has sent a wrong answer
  • on the final check reverse the filter (near line 313)
    if ((count($useranswers) == $ncorrect) and ($nhits == $ncorrect)) { $correctanswer = true; $response = $correctresponse; $newpageid = $correctpageid; } else { $response = $wrongresponse; $newpageid = $wrongpageid; }
    will then be

if($wronguseranswer) { // at least one useranswer is wrong $response = $wrongresponse; $newpageid = $wrongpageid; } else {
// no wrong answers and the user has provided at least one answer
if ($nhits!= $ncorrect) { $missinganswers=true; }
$correctanswer = true;
$response = $correctresponse;
$newpageid = $correctpageid;
}

On the result page the user then shoul be presented with a third type of report: "Answer provided are correct but incomplete"

Hope this helps

Activity

Hide
Christian Deligant added a comment -

There is a third issue as well!

If you select ALL the answers, you get 15 points as well!!! which is obviously wrong

Show
Christian Deligant added a comment - There is a third issue as well! If you select ALL the answers, you get 15 points as well!!! which is obviously wrong
Hide
Christian Deligant added a comment -

Sorry: the version in use is Moodle 1.9 Beta (2007081500)

Show
Christian Deligant added a comment - Sorry: the version in use is Moodle 1.9 Beta (2007081500)
Hide
Christian Deligant added a comment - - edited

I found out where the sum bug of the user's answers is!
in /mod/lesson/locallib.php, towards line 1453

the code
if ($lesson->custom) {
$attempt = end($attempts);
// If essay question, handle it, otherwise add to score
if ($pages[$attempt->pageid]->qtype == LESSON_ESSAY) { $essayinfo = unserialize($attempt->useranswer); $earned += $essayinfo->score; $nmanual++; $manualpoints += $answers[$attempt->answerid]->score; } else { $earned += $answers[$attempt->answerid]->score; }
} else {
...

should be modified as follow

if ($lesson->custom) {
$attempt = end($attempts);
// If essay question, handle it, otherwise add to score
if ($pages[$attempt->pageid]->qtype == LESSON_ESSAY) { $essayinfo = unserialize($attempt->useranswer); $earned += $essayinfo->score; $nmanual++; $manualpoints += $answers[$attempt->answerid]->score; } } elseif($attempt->correct) {
foreach(explode(',', $attempt->useranswer) as $localanswerid) { $earned += $answers[$localanswerid]->score; }
}
} else {
...

Hope this helps

Show
Christian Deligant added a comment - - edited I found out where the sum bug of the user's answers is! in /mod/lesson/locallib.php, towards line 1453 the code if ($lesson->custom) { $attempt = end($attempts); // If essay question, handle it, otherwise add to score if ($pages[$attempt->pageid]->qtype == LESSON_ESSAY) { $essayinfo = unserialize($attempt->useranswer); $earned += $essayinfo->score; $nmanual++; $manualpoints += $answers[$attempt->answerid]->score; } else { $earned += $answers[$attempt->answerid]->score; } } else { ... should be modified as follow if ($lesson->custom) { $attempt = end($attempts); // If essay question, handle it, otherwise add to score if ($pages[$attempt->pageid]->qtype == LESSON_ESSAY) { $essayinfo = unserialize($attempt->useranswer); $earned += $essayinfo->score; $nmanual++; $manualpoints += $answers[$attempt->answerid]->score; } } elseif($attempt->correct) { foreach(explode(',', $attempt->useranswer) as $localanswerid) { $earned += $answers[$localanswerid]->score; } } } else { ... Hope this helps
Hide
Mark Nielsen added a comment -

The primary problem with the Multiple choice question with multi answer is that the form is not tailored for how the question is actually handled. The lesson questions are very basic and are all or nothing. So, the recommended method for constructing this type of question is to have all correct answers first, followed by the incorrect answers. Then set all correct answers to have the same score and jump settings and do the same with all incorrect answers. Then when the question is graded and you have the correct answer, you get the score and jump value of one of the correct answers and if you have the incorrect answer you get the score and jump value of one of the incorrect answers.

I don't necessarily disagree with your modifications, but they do drastically change how the question is graded and would be a rather nasty surprise for current lesson users.

With all that said, there is a grading bug and I have a proposed fix attached. When multi answer is used, the question stores the incorrect answer ID in the attempt and thus uses the wrong answer for grading. If you want to try it out and see if it works that would be great. If I don't hear from you soon then I'll put it in the code and hope for the best.

Show
Mark Nielsen added a comment - The primary problem with the Multiple choice question with multi answer is that the form is not tailored for how the question is actually handled. The lesson questions are very basic and are all or nothing. So, the recommended method for constructing this type of question is to have all correct answers first, followed by the incorrect answers. Then set all correct answers to have the same score and jump settings and do the same with all incorrect answers. Then when the question is graded and you have the correct answer, you get the score and jump value of one of the correct answers and if you have the incorrect answer you get the score and jump value of one of the incorrect answers. I don't necessarily disagree with your modifications, but they do drastically change how the question is graded and would be a rather nasty surprise for current lesson users. With all that said, there is a grading bug and I have a proposed fix attached. When multi answer is used, the question stores the incorrect answer ID in the attempt and thus uses the wrong answer for grading. If you want to try it out and see if it works that would be great. If I don't hear from you soon then I'll put it in the code and hope for the best.
Hide
Christian Deligant added a comment -

I saw the patch and it seems OK.

I understand the issue about current students, so I will build a new lesson module that applies the logic I described, that is

  • multiple choice question with more than one correct answer
  • the question has a (maxvalue) field
  • each correct answer has a positive value, wrong answers have 0
  • the score of the attempt is the lower value of (summ of the answers given, but 0 if there is one wrong answer) and (maxvalue)
  • the grading is evaluated from (sum of the scores of the user'sanswers)/(sum of the question's maxvalues)

in this way, a teacher can make questions like the one given, and the student can't cheat answering all the answers

Hope this helps

PS: let me know if the module can be of interest

Show
Christian Deligant added a comment - I saw the patch and it seems OK. I understand the issue about current students, so I will build a new lesson module that applies the logic I described, that is
  • multiple choice question with more than one correct answer
  • the question has a (maxvalue) field
  • each correct answer has a positive value, wrong answers have 0
  • the score of the attempt is the lower value of (summ of the answers given, but 0 if there is one wrong answer) and (maxvalue)
  • the grading is evaluated from (sum of the scores of the user'sanswers)/(sum of the question's maxvalues)
in this way, a teacher can make questions like the one given, and the student can't cheat answering all the answers Hope this helps PS: let me know if the module can be of interest
Hide
Mark Nielsen added a comment -

Applied patch.

Show
Mark Nielsen added a comment - Applied patch.

People

Vote (0)
Watch (1)

Dates

  • Created:
    Updated:
    Resolved: