Details
-
Type:
Bug
-
Status: Closed
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: 2.5.2
-
Component/s: Workshop
-
Labels:
-
Testing Instructions:
-
Affected Branches:MOODLE_25_STABLE
-
Fixed Branches:MOODLE_24_STABLE, MOODLE_25_STABLE, MOODLE_26_STABLE
-
Pull from Repository:
-
Pull Master Branch:
MDL-41954-workshop-zero-variance
Description
This was observed by Helen Foster while she was evaluating the MOOC workshop today at learn.moodle.net site. The following description is slightly simplified from the real situation but it illustrates the situation well (and could be used as steps to reproduce).
STEPS TO REPRODUCE
- There is a workshop, using the grading strategy "Number of errors".
- The assessment form has three assertions (dimensions), all with the same weight 1.
- There was a submission made by a participant Alice that was peer-reviewed by three other participants Bob, Cindy and Dan.
- Bob assessed the submission and filled the assessment form with No, No, No.
- Both Cindy and Dan assessed the submission and filled the assessment form with Yes, Yes, Yes.
- Helen (as the workshop facilitator/teacher) used the grading evaluation method "Comparison with the best assessment" during the grading evaluation phase to calculate the final grades.
- Helen realized that Bob's assessment is wrong and that Cindy and Dan are right. So she set the weight of Bob's assessment to 0 (zero) and re-calculated grades.
EXPECTED BEHAVIOUR
Bob should get very low grade for this assessment and both Cindy and Dan should get full grade for their assessments.
WHAT ACTUALLY HAPPENED
Bob got the full grade for his assessment, too.
Gliffy Diagrams
Activity
Submitting for integration. The submitted fix makes the code handle the calculation even in case of the dimension assessments variance lower than 0.01 by assuming it as the lowest reasonable value.
Thanks David, integrated in 2.4, 2.5 and master
Passing, thanks!
You did it!
Thanks for your contribution, this change has been merged upstream and is now available on your local git mirror and on download sites shortly.
Good catch! There is a bug in the mod/workshop/eval/best/lib.php library. It affects the calculation of the grade for assessment in this specific situation only, luckily. What happened here is that, thanks to all the coincidences, the calculated variance of received assessments was zero in all three assertions (because Bob's assessment did not count and all others assessments were equal). And the situation of zero variance is not covered well in the code. The line
if ($var > 0.01 and $agrade != $rgrade) {caused that the "distance" of Bob's assessment from the "best" (referential) one is not calculated if the variance is zero.
I wrote a unit test that confirms this bug and applied a simple fix (that should had been here since ever).