Moodle
  1. Moodle
  2. MDL-22423

SCORM resource type asset should not be included in gradebook calculations

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 1.9.8
    • Fix Version/s: 1.9.10
    • Component/s: SCORM
    • Labels:
      None
    • Environment:
      LAMP, 1.9.8, not browser-specific. sample package attached.
    • Database:
      MySQL
    • Affected Branches:
      MOODLE_19_STABLE
    • Fixed Branches:
      MOODLE_19_STABLE
    • Rank:
      26874

      Description

      When the gradebook grade for a multi-sco scorm package is calculated, if the grading method is set to Average, any asset in the content structure is included in this calculation, even though assets are technically not scos. This lowers the average for the final grade

      A similar issue occurs when grading method is learning objects. In this case, since higher than 100% of scos have lesson_status of 'completed', the grade is higher than 100 in the gradebook.

      Can we institute a check to see whether a resource is an asset or a sco before we calculate gradebook grades, and exclude assets from the calculations?

      Forum discussion here: http://moodle.org/mod/forum/discuss.php?d=149876#p655872

        Issue Links

          Activity

          Hide
          Dan Marsden added a comment -

          Hi Amy, thanks for the report!

          makes sense to me! - happy to review a patch if someone else comes up with one, otherwise this will likely need to wait a while.

          Show
          Dan Marsden added a comment - Hi Amy, thanks for the report! makes sense to me! - happy to review a patch if someone else comes up with one, otherwise this will likely need to wait a while.
          Hide
          Piers Harding added a comment -

          Hi -

          I've had a look at this, and there are a few things that make this tricky.

          Firstly, even though your scorm package sets the cmi.core.lesson_status to 'browsed' for assets, mod/scorm forces the status to 'completed' for all 'asset' type scos.
          If the asset type scos were left at browsed, then they would not be included in the scoring. But, the overall completion state for the package could be left 'incomplete' and ungraded (this is probably why mod/scorm forces them).

          Alternatively, if we were to exclude 'asset' type scos from scoring, then users out there with 'asset' only packages, that relied on each sco being visited (not a score) eg. learning objects counted, then this would break grading for them.

          Given this, I see this more as a feature request to 'ignore assets' or possibly only ignore assets if the grading method is compatible?

          Any comments on this from anyone?

          Cheers,
          Piers Harding.

          Show
          Piers Harding added a comment - Hi - I've had a look at this, and there are a few things that make this tricky. Firstly, even though your scorm package sets the cmi.core.lesson_status to 'browsed' for assets, mod/scorm forces the status to 'completed' for all 'asset' type scos. If the asset type scos were left at browsed, then they would not be included in the scoring. But, the overall completion state for the package could be left 'incomplete' and ungraded (this is probably why mod/scorm forces them). Alternatively, if we were to exclude 'asset' type scos from scoring, then users out there with 'asset' only packages, that relied on each sco being visited (not a score) eg. learning objects counted, then this would break grading for them. Given this, I see this more as a feature request to 'ignore assets' or possibly only ignore assets if the grading method is compatible? Any comments on this from anyone? Cheers, Piers Harding.
          Hide
          Dan Marsden added a comment -

          talking with Piers - it looks like there might be 2 related bugs here.....

          learning objects returning higher than 100% - Amy can you please provide more info on this one?

          Average including both assets and SCO's in the calculation

          I'm tempted to suggest that we create a new grading method "Average of SCO's" - I'm weary of changing the "expected" behaviour of existing SCORM objects - people may have become used to Moodle returning an average including Assets in the grade - does that make sense to everyone?

          Show
          Dan Marsden added a comment - talking with Piers - it looks like there might be 2 related bugs here..... learning objects returning higher than 100% - Amy can you please provide more info on this one? Average including both assets and SCO's in the calculation I'm tempted to suggest that we create a new grading method "Average of SCO's" - I'm weary of changing the "expected" behaviour of existing SCORM objects - people may have become used to Moodle returning an average including Assets in the grade - does that make sense to everyone?
          Hide
          Amy Groshek added a comment -

          Hi Piers and Dan,

          With regard to the comment about scorm activities with grading set to learning objects, I do not know how the code is processing this information, so I can't determine for sure whether it's the same or a related bug. In both cases it is a matter of whether or not assets are included in the calculation of the gradebook score.

          Normally, with a multi-sco scorm package, if grading is set to learning object, then the score in the gradebook is a average of the completed learning objects, with 100% meaning that all scos are completed. So if there are 4 SCOs, all completed, the gradebook total is 100%. What I have observed (or seem to have observed) is that in the calculation of that %, assets are included in the calculation of the numerator total, but not in the calculation of the denominator. The result is a scary looking sort of percent in the gradebook column.

          This is the a typical scenario:
          SCORM package, 4 scos, all completed, gradebook grade = 100%

          This is the scenario I have observed:
          SCORM package, 3 scos, all completed, 1 asset, browsed, gradebook grade = 133%

          Alternatively, if we were to exclude 'asset' type scos from scoring, then users out there with 'asset' only packages, that relied on each sco being visited (not a score) eg. learning objects counted, then this would break grading for them.

          Piers, I see your concern here. Is there anything in the scorm standard that specifies how this should work. It's always far easier to explain what a customer sees as aberrant behavior if it is also conformant behavior. If I get time this week I will look this up.

          Show
          Amy Groshek added a comment - Hi Piers and Dan, With regard to the comment about scorm activities with grading set to learning objects, I do not know how the code is processing this information, so I can't determine for sure whether it's the same or a related bug. In both cases it is a matter of whether or not assets are included in the calculation of the gradebook score. Normally, with a multi-sco scorm package, if grading is set to learning object , then the score in the gradebook is a average of the completed learning objects, with 100% meaning that all scos are completed. So if there are 4 SCOs, all completed, the gradebook total is 100%. What I have observed (or seem to have observed) is that in the calculation of that %, assets are included in the calculation of the numerator total, but not in the calculation of the denominator. The result is a scary looking sort of percent in the gradebook column. This is the a typical scenario: SCORM package, 4 scos, all completed, gradebook grade = 100% This is the scenario I have observed: SCORM package, 3 scos, all completed, 1 asset, browsed, gradebook grade = 133% Alternatively, if we were to exclude 'asset' type scos from scoring, then users out there with 'asset' only packages, that relied on each sco being visited (not a score) eg. learning objects counted, then this would break grading for them. Piers, I see your concern here. Is there anything in the scorm standard that specifies how this should work. It's always far easier to explain what a customer sees as aberrant behavior if it is also conformant behavior. If I get time this week I will look this up.
          Hide
          Jim Edgar added a comment -

          Hi Dan,

          I'd just like to briefly respond to your comment about the average including assets and SCOs in the calculation.

          --------------------------------------------------------
          Average including both assets and SCO's in the calculation

          I'm tempted to suggest that we create a new grading method "Average of SCO's" - I'm weary of changing the "expected" behaviour of existing SCORM objects - people may have become used to Moodle returning an average including Assets in the grade - does that make sense to everyone?
          --------------------------------------------------------

          This "SCO-average" bug appeared in version 1.9.8. It was not an issue in 1.9.7, so there aren't likely too many users who have adapted to it. The only way one could effectively adapt to this, is by not having any assets other then the SCOs in a LO. That being the case, a fix to this bug should not inconvenience anyone using that workaround.

          To describe the issue more specifically, the problem occurs when playing a SCORM 1,2 package with both SCOs and nonSCOs in the main menu; each time one of these nonSCO menu items is displayed, Moodle treats it as a SCO and assigns it a score of zero. It includes these in the average attempt of the SCORM object. (I the nonSCO menu items aren't opened, they are not used to calculate the average of that attempt on the SCOs in the LO.)

          JIm

          Show
          Jim Edgar added a comment - Hi Dan, I'd just like to briefly respond to your comment about the average including assets and SCOs in the calculation. -------------------------------------------------------- Average including both assets and SCO's in the calculation I'm tempted to suggest that we create a new grading method "Average of SCO's" - I'm weary of changing the "expected" behaviour of existing SCORM objects - people may have become used to Moodle returning an average including Assets in the grade - does that make sense to everyone? -------------------------------------------------------- This "SCO-average" bug appeared in version 1.9.8. It was not an issue in 1.9.7, so there aren't likely too many users who have adapted to it. The only way one could effectively adapt to this, is by not having any assets other then the SCOs in a LO. That being the case, a fix to this bug should not inconvenience anyone using that workaround. To describe the issue more specifically, the problem occurs when playing a SCORM 1,2 package with both SCOs and nonSCOs in the main menu; each time one of these nonSCO menu items is displayed, Moodle treats it as a SCO and assigns it a score of zero. It includes these in the average attempt of the SCORM object. (I the nonSCO menu items aren't opened, they are not used to calculate the average of that attempt on the SCOs in the LO.) JIm
          Hide
          Dan Marsden added a comment -

          Thanks Jim! - I hadn't realised this bug was introduced so recently - It should be safe for us to fix the behaviour rather than create a new grading method

          Show
          Dan Marsden added a comment - Thanks Jim! - I hadn't realised this bug was introduced so recently - It should be safe for us to fix the behaviour rather than create a new grading method
          Hide
          Dan Marsden added a comment -

          just found the "cause" of this MDL-18857

          Show
          Dan Marsden added a comment - just found the "cause" of this MDL-18857
          Hide
          Jim Edgar added a comment -

          That is good news, Dan. Thanks for taking the time to look into this. I hope it is not a difficult thing to fix.

          Show
          Jim Edgar added a comment - That is good news, Dan. Thanks for taking the time to look into this. I hope it is not a difficult thing to fix.
          Hide
          Dan Marsden added a comment -

          something like this could work although I'm not convinced that it's the "right" fix -

          --- mod/scorm/locallib.php	25 Jun 2010 10:12:30 -0000	1.113
          +++ mod/scorm/locallib.php	29 Jun 2010 00:56:42 -0000
          @@ -515,7 +515,8 @@
                       if (($userdata->status == 'completed') || ($userdata->status == 'passed')) {
                           $attemptscore->scoes++;
                       }
          -            if (isset($userdata->score_raw)) {
          +            if (!empty($userdata->score_raw) || ($scorm->type=='sco' && isset($userdata->score_raw))) {
                           $attemptscore->values++;
                           $attemptscore->sum += $userdata->score_raw;
                           $attemptscore->max = ($userdata->score_raw > $attemptscore->max)?$userdata->score_raw:$attemptscore->max;
          
          Show
          Dan Marsden added a comment - something like this could work although I'm not convinced that it's the "right" fix - --- mod/scorm/locallib.php 25 Jun 2010 10:12:30 -0000 1.113 +++ mod/scorm/locallib.php 29 Jun 2010 00:56:42 -0000 @@ -515,7 +515,8 @@ if (($userdata->status == 'completed') || ($userdata->status == 'passed')) { $attemptscore->scoes++; } - if (isset($userdata->score_raw)) { + if (!empty($userdata->score_raw) || ($scorm->type=='sco' && isset($userdata->score_raw))) { $attemptscore->values++; $attemptscore->sum += $userdata->score_raw; $attemptscore->max = ($userdata->score_raw > $attemptscore->max)?$userdata->score_raw:$attemptscore->max;
          Hide
          Dan Marsden added a comment -

          fix now in HEAD and 1.9Stable

          I've gone for the fix I've posted above as it will reproduce the exact behaviour from before the commit for MDL-18857 but allow empty scores from sco's to be included in the result. - thanks for the report!

          Show
          Dan Marsden added a comment - fix now in HEAD and 1.9Stable I've gone for the fix I've posted above as it will reproduce the exact behaviour from before the commit for MDL-18857 but allow empty scores from sco's to be included in the result. - thanks for the report!
          Hide
          Jim Edgar added a comment -

          Dan,

          Great stuff! I've installed the fix and tested it. The fix is excellent!

          Thanks again,
          Jim

          Show
          Jim Edgar added a comment - Dan, Great stuff! I've installed the fix and tested it. The fix is excellent! Thanks again, Jim
          Hide
          Dan Marsden added a comment -

          thanks! - closing bug.

          Show
          Dan Marsden added a comment - thanks! - closing bug.

            People

            • Votes:
              3 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: