Moodle

This is a major issue and needs to be resolved for the sum aggregation to be useful.

We looked into the solution for drop lowest/keep highest and it involves a major rewrite of the grade libraries.

Exclusion does not seem fixable and may need to be removed from the sum aggregation method unless the grade items were somehow normalized based on the expected number of grades in a category.

Both of these present HUGE problems for Moodle.

Sum of grades also does not exclude items hidden in the gradebook from the grademax, or any empty items. This means any items that exist, but are not graded negatively affect the total grade.

okay, I've taken a first swipe at a patch for this. It appears to me that the changes are mostly localized to lib/grade/grade_category.php in the sum_grades function which is only called when sum of grades is selected. A few issues quickly show up when I attempt to do this.

1) How should grades be dropped when items within the category have different maximum points?

Assign1 = 8/10
Assign2 = 10/10
Assign3 = 22/100

If we drop 1 score, do we drop Assign1 or Assign3? Also, related, what is the maximum score for the category (20 or 110)? This is clearly a policy decision.

I've choosen to implement dropping Assign1 in this case and assigning this user a maximum of 110 points. Each user can have their own maximum for the category and things seem to be correct. I've also chosen to include scale graded items when dropping items, but not when tallying maximum or cumulative points.

This patch is probably very wrong, but it might jump start this bug into getting fixed. I'm open to review.

A first swipe for fix dropping of grades and maximum value in a sum_of_grades category.

This patch extends the patch by Paul Ortman. It correctly calculates course and category totals as well as course averages when using the Sum of Grades aggregation method. Exclusions and extra credit are supported.

The patch alters sum_grades() to save the total maximum points available for a user into the category's grade_grade object for that user. grade_format_gradevalue() is also altered to accept a grade_grade object as an optional parameter. For category and course totals, the grade_grade object is used to calculate the user's score instead of the possibly inaccurate grade_item object. The patch has not been tested on other aggregation methods.

These changes make the Sum of Grades aggregation method usable.

The item maximum grade must be the same for all students, this is one of the fundamental design rule we have in the current fradebook. It is not going to be changed, sorry.

The fundamental issue with fixing this is that any fix would have to make certain assumptions:

1: That entered grades are out of the same maximum value... not going to happen.

2: It's OK to normalize grades first in order to calculate which grade is to be dropped/kept... still screws students if grades are out of the same non-normalized category maximum grade.

3: It's OK to change the category total for each student based on the specific items dropped/kept/excluded.... breaks the entire idea of what the sum aggregation method is, also makes aggregating these category totals impossible.

4: It's OK to normalize the maximum category totals for each student to equal the unmolested sum category maximum... we've come full circle folks and have given you simple weighted mean and told you it was sum.

All four assumptions would have to be made (if the first one were not true, then it would need to be forced to be true) and all four assumptions are incorrect.

The BEST solution and the one we've implemented is to NOT ALLOW exclusions, drop lowest, or keep highest in the sum aggregation method. We also force all empty grades to be counted as zero in the sum aggregation method. This gives students a realistic expectation of grades and gives faculty a consistent application of a VERY common aggregation method.

As it stands in Moodle core, sum is broken if you use drop lowest, keep highest, ignore empty grades, and exceptions. This patch does not fix the problem (as it truly cannot be fixed without changing sum into simple weighted mean).

@Robert Russo – Yes, the patch I submitted was not meant to fix drop lowest, keep highest, or ignore empty grades. It does, however, work for grade exceptions and extra credit.

It's unfortunate the fundamental design rules of the gradebook won't be re-evaluated for such a basic need as user-individualized grading.

reopening, it is a good idea to prevent problematic exclusions, please not there is a separate bug fro drop low and drop high which is fixable.

to sum it up:

• extra credit - should work fine now, please note you might need to enable Unlimited grades
• fixing drop low and drop high right now
• exclusions are going to be forbidden/ignored for sum aggregation type

thanks for all the ideas and feedback

editing the title - keeping this issue for excluded only, the droppable grades are already fixed

I have reviewed the current code and the excluded item is just "excluded" from aggregation - the sum ignores it completely.
The prevention of excludes would probably cause more problems and would be hard to explain/document.

Closing as not a bug because the sum max value must be the same for all students.

In any case thanks for the report and all feedback

Confirming that this is not a bug but a feature

Hi,
I have just tested exclusions on testing environment and they give me the same results regardless if I choose Sum or Simple Weighted. Please see attached screenshot. Aggregation there is set to Simple Weighted. I believe all three students should get the same amount of points/percentage 205.00 (57.75 %) in this situation. Should not they?

P.s. I do not think it will be hard to explain or document why Sum does not allow/show exclusions. I believe it is vice versa - it is harder to explain why some feature is there and "does not work". %)

This all depends on how you define "Excluded", in current gradebook it means the grade item of the excluded grade is no considered in calculation.

The simple weight and sum are hacks that I created later in order to satisfy common requests. Unfortunately the Excluded can not work the same way for them.
1/ in case of simple mean it kind of breaks the points concept
2/ in case of sum Excluded is equivalent to no grade

My advice is simple, if you use simple mean or sum aggregation do not use Excluded grades.

Technically this can not be changed because the grade item properties (min, max, pass, etc.) must be the same for all users - if I understand it correctly people want the excluded grade to modify the max values of category item only for some users.

Petr,

Normally I would agree with you, but explaining this to 1700 faculty members and defining all the exclusions to the rules seemed like we would be setting ourselves up for future support calls.

What we did to combat this, was to remove exclude from the sum aggregation method, and scale the newly changed category totals to the theoretical max in SWM, giving faculty the predictability they wanted without making them memorize even more exceptions to the many rules in the Moodle gradebook.

I am sorry, but I think that this feature was working just fine in 1.9.4. And now it will stop working in 1.9.5? I do not think that people will be thrilled, if this is the case.
Robert, I think we want your code.

It is very intuitive for people (with whom I work here ) to be able to use sum/simple weighted and use exclusions. I gave up hope on Sum, but I really wish for at least Simple Weighted to work and work simply.
Petr, you just said that you have introduced those aggregations as hacks based on the common requests. Having Excludes to work there is also a common feature that people want.

The problem is "excluded" is being used to mean different things in different contexts.

"...in current gradebook it means the grade item of the excluded grade is no considered in calculation." Even this is not completely correct--You mean "It is not considered in the calculation to determine the value accumulated by the student, but it is still used for the calculation of the maximum value of the course."

This is an extremely confusing use of the "excluded" and effectively makes "excluded" grades the same as 0s. I'm wondering under what circumstances it would be desirable and preferred to grade an assignment with a non-zero grade then to "exclude" the grade instead of simply assigning a grade of 0 in the first place.

I understand from a code point of view why the gradebook operates this way, but it is completely backward to what most people expect. "Excluded" to most people means "Ignore this grade for all purposes."

"Technically this can not be changed because the grade item properties (min, max, pass, etc.) must be the same for all users - if I understand it correctly people want the excluded grade to modify the max values of category item only for some users."

This is a flaw in the gradebook core and needs to be re-evaluated. Is this the correct place to file bugs against the gradebook, or does this only concern gradebook reports? If the latter is true, where and to whom should bug reports on the gradebook itself be filed?

I think Brett has worded it very nicely. For me:
Exclude student from a grade = give him 100% for it towards the Total in case he was sick. But I still do not want to actually give him the real grade.
If Exclude = 0%, then it does not have any value in it (from my perspective), since students can be already either not-graded at all or given 0%.

P.s. I have re-read what Brett wrote, and we are talking about different things in terms of the second part ("Technically this can not be changed because the grade item properties (min, max, pass, etc.) must be the same for all users - if I understand it correctly people want the excluded grade to modify the max values of category item only for some users." This is a flaw in the gradebook core and needs to be re-evaluated.") I do not have any option on this for now, as I am happy with grades going over 100%
But I think we are still on the same page with what Exclude means.

p.s. This is how exclude works in current1.9.4 both for sum and simple weighted, and this what I would normally expect.

hmm, so you are proposing that Excluded grades are considered as 100% in case of Sum and Simple mean? Did anybody say that before?
Technically this would be possible in case of sum and simple mean

reopening, going to discuss it with more people...

I tried implementing this solution before I implemented recalculating each category on a per-student basis. The problem is that if you award 100% of the grade for excluded grades, you can run into a situation where a student receives credit in a category where he should have none. Consider the following:

Grade 1: 0/100
Grade 2: 0/100
Grade 3: 0/100 Excluded. (So aggregated as 100/100)

In this case the student would receive 100/300 points, giving him a category grade of 33%. For our installation we needed excluded grades to be completely ignored--The student would receive 0/200 for a category grade. If this is not a requirement, then implementing excluded grades to award 100% points might be a possibility, but I still feel it is more of a hack than recalculating based upon what student grades are actually used in the aggregation of his full points.

100% doesn't make sense for SWM as it artificially inflates scores.

Our idea is as so:
90/100 40/50 and 10/20 = 140/170 = 82.35
Here should be what happens when you exclude the 3rd grade:
90/100 40/50 and 10/20 = 130/150 = x/170 = 147.33/170 = 86.67

Here is what happens if you make the missing grade 100%:
90/100 40/50 and 20/20 = 150/170 = 88.24

You can see that there is some overinflation in the last example, whereas the second example is true to the 130/150 actual grade, it simply removes the need for the 3rd grade. This is the essence of what "exclude" is.

Bret gave a more egregious example showing the overinflation problem.

Using his example, our method would be as follows.

0/100 0/100 0/100 (excluded) = 0/200 = 0/300 = 0

Gotcha! You are right, I was oversimplifying

I am still a bit confused :-D

===============
SWM:
"Our idea is as so:
90/100 40/50 and 10/20 = 140/170 = 82.35
Here should be what happens when you exclude the 3rd grade:
90/100 40/50 and 10/20 = 130/150 = x/170 = 147.33/170 = 86.67"

I suppose I could make it work this way, give me a few days

===============
SUM:

could somebody give some examples?

Without making SUM into SWM, there is no real way to do exclusions in SUM.

That's why we omit the exclude option when their aggregation method is SUM.

"Our idea is as so:
90/100 40/50 and 10/20 = 140/170 = 82.35
Here should be what happens when you exclude the 3rd grade:
90/100 40/50 and 10/20 = 130/150 = x/170 = 147.33/170 = 86.67"

The grade is actually 130/150 or 86.67, but we scale the grade up accordingly to 147.33/170, not really changing the calculated value, but allowing for the 170 maximum possible points in the category total so as not to upset Moodle.

BTW. We love you Petr!

Please correct me if I'm wrong, but SUM and SWM are fundamentally different as far as the "weight" grades have by default.

Sum:
Grade 1: 10/10
Grade 2: 75/100
Grade 3: 0/10
Final Grade: 10/10 + 75/100 + 0/10 = 85/120 = 0.7083 = 71%

SWM (Defaults to a weight of 1):
Grade 1: 10/10
Grade 2: 75/100
Grade 3: 0/10
Final Grade: (1.00 + 0.75 + 0) / 3 = 0.583 = 58%

Two very different grades. Yes, there are options to change the weights in SWM, but as Robert said, try explaining this to 1700 teachers that want it to "just work." Having to change the weight of a 10 point quiz so it's not worth the same as a 100 point exam is very unintuitive for most people.

There are also, if I remember correctly, problems with extra credit and SWM. I can't remember the details, but will investigate further if anyone is interested...

Hi Brett, it would be great!

This is what one of my co-workers and faculty has to say, and he agrees with a suggestion from Robert for SWM (I can only transmit, since I am lost myself )

In a course where there are no categories and straight points (SUM) are used the same theory could apply. The difference is in the logic, not the end calculation. My thinking is that dropping a grade means the points for that activity are dropped from the total points on a per student basis.

Grade 1: 15/20
Grade 2: 40/50
Grade 3: 60/100
Grade 4: 0/20
Total points: 190
Grade: 115/190 = 60.52%

Now, I drop the missed activity:

Grade 1: 15/20
Grade 2: 40/50
Grade 3: 60/100
Grade 4: 0/20 - dropped
Total points: 170
Grade: 115/170 = 67.65%

What essentially the suggestion is that we assigned a grade to the excluded item based on the weighted percentage of the rest of the activities:

Grade 1: 15/20
Grade 2: 40/50
Grade 3: 60/100
Grade 4: 13.53/20
Total points: 190
Grade: 128.53/190 = 67.65%

@Elena – This is exactly what I was implementing!

From a programming point of view this is a little messy, but much less messy that recalculating the category sum each time through.

Petr, Robert: Thoughts?

Brett,

Your example for SWM is really Mean (dividing the unweighted grade quotient by the number of items) instead of SWM (the points weighted grade quotient divided by the total possible points).

We have implemented exclusions and extra credit in the SWM and WM aggregation methods in the LSU grade book about a year ago, but we had to make some fundamental changes to the default aggregation coefficient in order to avoid adding an extra extra credit flag in the database...we were too aggressive trying to do this without core changes, which was overzealous on our part.

Our next iteration will be simpler and easier for developers to understand and hopefully integrate into core more seamlessly.

Hi Folks,

Elena got me into this issue this morning. Robert I think clarified what I was confused about in Brett's example.

"Moodle calls this Aggregation method of computing course grades Simple weighted mean of grades because items are weighted based simply on their maximum point value."

I assume this is SWM, or what I call straight points. SUM would also fit the description, sum of all points earned divided by the sum of total points possible.

"Moodle calls this Aggregation method of computing course grades Weighted mean of grades because the course grade is computed by weights that the teacher assigns to categories (or items). "

I assume this is WM.

Brett's example:

SWM (Defaults to a weight of 1):
Grade 1: 10/10
Grade 2: 75/100
Grade 3: 0/10
Final Grade: (1.00 + 0.75 + 0) / 3 = 0.583 = 58%

To me appears to be Mean, as Robert suggests:

"Moodle calls this Aggregation method of computing grades Mean of grades because the average is computed by giving the same weight to each item. '

Brett states: "From a programming point of view this is a little messy, but much less messy that recalculating the category sum each time through."

It would be messy in Excel. I can't imagine it in php. I took a few semesters of VB in about 1994 and can copy and paste javascript into HTML and make some things work.

Now: My question. In moodle docs it states:

"Note that in version 1.9, by default, items are not computed into the grade until a score is assigned. This means that you can post your activities as far in advance as you wish; students always see a grade based only on what has been marked. You can change this to have all non-graded items be counted as zero if you wish. "

I'm probably way off base and know nothing about how this feature is implemented in code, but could an excluded item simply "press that button" which appears to already exist?

@Robert,

Could you please provide an example of the calculation? The part I'm confused about is "the points weighted grade quotient." Isn't the weight configurable per grade item and defaults to 1?

Here is an expanded version of how I thought I understood SWM:

SWM (where default weight = 1)
Grade 1: 10/10 = 1 * weight = 1
Grade 2: 75/100 = 0.75 * weight = 0.75
Grade 3: 0/10 = 0.00 * weight = 0.00
Final Grade: (1.00 + 0.75 + 0) / 3 = 0.583 = 58%

Are you suggesting this calculation is correct?:
SWM (where default weight = 1)
Grade 1: 10/10 = 1
Grade 2: 75/100 = 0.75
Grade 3: 0/10 = 0.00
Final Calculation: (1.00 + 0.75 + 0) * weight = 1.75.
Final Grade: 1.75 / 3 = 0.583 = 58%

I just wanted to add that I've got a note from yet another coordinate campus and they agree on Robert's calculations for SWM.
(overall I feel ashamed, as 3 years of higher algebra were apparently worthless to me %))

nice - so now we have general agreement on SWM + excluded grades

Can we also define excluded for SUM, or should it be completely prevented? Hmm, there might be some backwards compatibility problems...

Will suggestion from Steve work?
http://tracker.moodle.org/browse/MDL-14831?focusedCommentId=69628&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_69628 - it's about SUM

Elena,

We brainstormed implementing "faking" the excluded grade with the mean of the category. It will work only if the final category grade is calculated as a percentage. If the final grade is shown as points, it will be artificially inflated. Consider:

Grade 1: 15/20
Grade 2: 40/50
Grade 3: 60/100
Grade 4: 13.53/20
Final Grade: 128.53 points

The student's points for the final points grade is artificially inflated--he has truly only earned 115 points. This isn't at all a problem if final grades are averages, but if a teacher is using raw points, this could be confusing...

ehhh
I would agree with everyone here, since I do not really get it .
I mean I can understand how do the math at the end, but I do not have an opinion why it should be one way or another.
Brett, in you example, are you talking about Mean, Sum, SWM or WM? I will run it to our faculty.

I just tested this examples in current cvs (no aggregation modifications since the last fix) and to my surprise the results were following :-O

Our idea is as so:
90/100 40/50 and 10/20 = 140/170 = 82.35%
Here should be what happens when you exclude the 3rd grade:
90/100 40/50 and (10/20 excluded) = 86.67%

The I tried
Grade 1: 15/20 Grade 2: 40/50 Grade 3: 60/100 Grade 4: 0/20 = Grade: 60.53%

Grade 1: 15/20 Grade 2: 40/50 Grade 3: 60/100 (Grade 4: 0/20 - dropped) = 67.65%

So it seems that after my last changes the SWM agg works as expected, am I right??

Apologies if this feedback may be a little bit late to the game. Having read all of this through and having worked out calculations in Excel for each of the aggregation types as part of my work on our upgrade from Moodle 1.8.5 with Gradebook Plus V2 to this new 1.9.4, I would agree / reiterate as follows:

Brett's example could be used for either SUM or SWM, right? BUT it should only be used for SWM, WM or Mean because those are all converted to grade percentages or normalized.

SUM
1. Let's not attempt to extend functionality to allow grade exclusion - because:
a) Don't we want to keep it simple for the SUM users? (Wasn't this the original intention?) If a grader wants to use Exclusion of individual grades we can point them to one of the other grading methods that normalizes grades before calculating points. (The other methods all do this, right?)
b) I imagine that SUM is useful for folks who do not wish to account for any implied calculations, conversions, or normalization of grades. They do not use a "Minimum grade" for any of their grade items. Every grade item's maximum points already represent the "weight" or relative importance for that item toward the course total. Chances are they don't need subcategories with varied aggregations. For many of these folks I see that their maximum grades for all items add up to 100. So they can add up each student's earned points and if the total for a student is 98 points that's the same as 98% and that's an A. Done.
b) Don't we want to avoid backward compatibility headaches? I can imagine having to tell graders that became accustomed to the simplicity of SUM in 1.9.4 that excluding a grade is now allowed for that method (in 1.9.5). It would require a lengthy explanation.

SWM
1. Doesn't this method allow for a grader to assign minimum points for a graded item?
2. Some kind of conversion/ normalization is needed to aggregate points/percentage?
2. Exclusion of a grade that involves some implied grade assignment is in keeping with other conversions needed with other choices the grader has made (such as minimum points).

Is this way off the mark?

I, personally , would also vote for keeping SUM simple, but may be still allow exclusions, and for SUM make Exclude = Excused = 100% for a grade. Yep, instructor can simply type it in, but the it is more fancier that way
P.s. I saw use of SUM a here lot too, but with bunch of categories for weeks, and the total is usually not 100 points at the end. More often it is something like 350 or 500.

@Petr – "So it seems that after my last changes the SWM agg works as expected, am I right??" Yes!!

@Elena – I was talking about SUM in this post : http://tracker.moodle.org/browse/MDL-14831?focusedCommentId=69703&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_69703

Giving 100% credit for a grade doesn't work because of artificial grade inflation, as outlined above.

@Kathy – There are a few inconsistencies and problems with your description.

SUM:
b) "Every grade item's maximum points already represent the 'weight...'" T
--This is correct, but the next part is a dangerous assumption:
"Chances are they don't need subcategories with varied aggregations. For many of these folks I see that their maximum grades for all items add up to 100. So they can add up each student's earned points and if the total for a student is 98 points that's the same as 98% and that's an A. Done."
--This is impossible to know and dangerous to assume teachers use it like this. The gradebook needs to be abstracted out so that it doesn't require any assumptions to work. We have many teachers that use max points as relative weight, but the final course points add up to well over 1000. For example, homework is worth 10 points, quizzes are worth 25, midterms are worth 100, and the final is worth 250 points. In a 18 week course with 3 midterms and a final, this will add up quickly to more than 100 points.

b.2) "Don't we want to avoid backward compatibility headaches? I can imagine having to tell graders that became accustomed to the simplicity of SUM in 1.9.4 that excluding a grade is now allowed for that method (in 1.9.5). It would require a lengthy explanation."
– The explanation doesn't need to be lengthy. The exclude option is already in SUM, it's just it doesn't work as most people except. If they're already used to using SUM without exclusions, telling them not to check the "Excluded" checkbox would be all the explanation they'd need.

I am considering adding warning to Exclude option if Sum aggregation selected in parent category - something like "Warning, excluding of grades in not compatible with sum aggregation.".

Does it make sense?

If we are not going to have Exclude in Sum, then, yes, having a warning will be super.

On our campus, our programmers have added a small warning in the range field for the total of the category set to SUM OF GRADES that states: "Range May Vary {second line} Dropped 1 Lowest Grade(s)"

The sum of grades functionality with dropping the lowest # technically exists already in the system, it's just a matter of letting users know how it works.

@Petr – At this point I agree. I would love to see more overall flexibility in the gradebook, but I understand the need to get 1.9.5 out. Here's hoping for a good look and code review at the gradebook for 2.0!

resolving as fixed, sum grades have a warning, SWM should be working as expected.

Thanks everybody, it was nice to work with all of you
Please file separate bugs for regressions or any new suggestions.

Since it seems to be related for me, I am a posting here
Can you please take a look at this screenshot for category total?
I have used SWM and checked "Aggregate only non-empty grades".
You may notice that total Points (not the %) for the third student looks misleading. (WM has the same issue).

I just want to confirm, that this is how it is supposed to be, and that I should recommend to display percents only in the total column, if such method is being used?

Thank you!

This is correct Elena.

What is happening with SWM is this:

50+30=80 out of 80 possible (due to the empty grade not being aggregated) = 80/80 = 1 = 100%
In order to make sure we don't have different category totals for raw points the grade is normalized to the category max of 100pts by multiplying the achieved grade by 100/80 or 1.25.

If this student had less than perfect grades, say 40+25 = 65/80 = 81.25% = 65 * 1.25 = 81.25pts

If you category total was 125 the calculations are as follows for my example:
40 + 25 = 65/80 = 81.25% = 65 * 1.5625 = 101.5625pts

Weighted mean functions in a similar way, except the grade quotients are added together and then multiplied by their weights and divided by the sum of the weights included in the calculations.

Ex:

40/50 = .8 weight 3
25/30 = .83333 weight 5
-/20 = - weight 2

This would be calculated as so:
.8 * 3 = 2.4
.83333 * 5 = 4.16666
2.4 + 4.16666 = 6.56666
6.56666 / 8 (the included weights) = 82.0833% or 102.60416 pts out of the above 125 max.

I hope this clears things up a bit...I may have made things worse in retrospect.

Thank you so much, Robert!

p.s. omg, I've got it!