Details

Type: Subtask

Status: Closed

Priority: Minor

Resolution: Fixed

Affects Version/s: 2.1

Fix Version/s: 2.1

Component/s: Questions

Labels:None

Testing Instructions:Hide
Create one course with language forced to French, and another with it forced to English.
1. Create numerical questions with various combinations of unit options and answers, in both courses.
2. Add them to a quiz and attempt it as students. Try typing a variety of answersShowCreate one course with language forced to French, and another with it forced to English. 1. Create numerical questions with various combinations of unit options and answers, in both courses. 2. Add them to a quiz and attempt it as students. Try typing a variety of answers 
Affected Branches:MOODLE_21_STABLE

Fixed Branches:MOODLE_21_STABLE

Pull from Repository:

Pull Master Branch:

Pull Master Diff URL:
Description
Gliffy Diagrams

 answer_resp.jpg
 25 kB

 grading_decsep.jpg
 18 kB

 grading_numerical3.jpg
 11 kB

 grading)numerical2.jpg
 30 kB
Issue Links
 has been marked as being related by

MDL27868 Hints get lost when creating calculated and calculatedmulti questions
 Closed

MDL28118 Think again about the meaning or 'right number, wrong unit' in qtype_numerical
 Open

MDL28138 format_float function in question/type/calculated/db/upgradelib.php seems broken
 Closed
 is duplicated by

MDL27363 Numerical question format and unit handling
 Closed
Activity
 All
 Comments
 Work Log
 History
 Activity
 Links Hierarchy
Tim,
If the conversion of a float to a text can change from one locale to another locale as PHP internationalization is a new moodle 2,1 requirement, when we want to export a given numerical answer from one locale to another locale, how should we do it ?
1If the answer is stored as digital i.e. using a localeindependent binary representation, we could use CDATA.
2 However answer are stored in a text field.
a This will mean that we need to convert from text to a locale independent binary representation on export, use CDATA and do the inverse process on import.
b Or we export the locale with the answer text so that on import the conversion could be done if necessary, to the import locale.
3 Another option could be that we don't use the general answer field for numerical but an additional field in numerical_options where we already store tolerance.
4or I don't understand well the problem
I will experiment by setting back the 2,0 apply_unit() for storing answer and an additional answer_numeric field in question_numerical to store the answer as number in addition to the answer regular text field of question_answers to detect any difference when working with different locales either in moodle language or PHP settings.
More news in 23 days as this imply mastering PHP locale settings in the beautiful Windows universe...
As background info from wikipwedia http://en.wikipedia.org/wiki/Decimal_mark#Examples_of_use
The following examples show the decimal mark and the thousands separator; the lists are ordered chronologically, by when each country adopted the use:
In Albania, Serbia[citation needed], Bosnia, Estonia, France, Finland, Hungary, Poland, Czech Republic, Slovakia and much of Latin Europe as well as French Canada: 1 234 567,89
In Germany, Netherlands, Belgium, Denmark, Italy, Portugal, Romania, Sweden, Slovenia and much of Europe: 1 234 567,89 or 1.234.567,89.
In Switzerland: For other values the SI style 1 234 567,89 is used with a "," as decimal symbol.
In Australia, English Canada, Israel, Japan, Korea (both), Malaysia, Singapore, the Philippines, the United Kingdom, and the United States: 1,234,567.89
In Croatia, SI style (1 234 567.89) is used.
SI style: 1 234 567.89 or 1 234 567,89 (in their own publications the dot is used in the English version and the comma in the French version).
In China, comma and space are used to mark digit groups because dot is used as decimal mark. There is no universal convention on digit grouping, so both thousands grouping and no digit grouping can be found. However, grouping can also be done every four digits: 123,4567.89, since names for large numbers in Chinese are based on powers of 10,000 (e.g. the next new word is for 108). Japan is similar.
In Mexico: 1'234,567.89; for million separator an apostrophe is used.
The variety of grouping schema for numbers greater than 1000 explain why the code try to eliminate these characters so that the final number is a serie of numbers before the decimal point which is either a . or a ,
So to store correctly the answer numerical value once the decimal has been identified anything at left should be only + and digitals.
 If there is a . with no , decimal_sep should be .
This is the case of answer value that are from the edit_numberical_form.php that has past the is_numeric() test.  If there is a . with , the decimal_sep is the sign at right (higher position in the string).
 If there is only , then it is either a correctly written number with , as decimal_sep or a > 1000 number written without a decimal point.we should choose the first solution i.e , as decimal_sep
You are doing good analysis.
However, I cannot agree with the proposal you make (points 13). Let me give just one specific example:
A. If an English OU student on an accounting course types 13,000 as the answer to a numerical question, then we must interpret that as 13000.
However, there are other equally important examples, for example,
B. If a French Canadian student answers the question 8,1234 g Iron Oxide and 4,8767 g Aluminium are placed in a crucible, what is the weight of the reactants? Give your answer to 5 significant figures, they should be able to just type 13,000, and we must interpret that as 13.000 (in PHP notation).
It is very hard for Moodle to get both of those examples right. When I was trying to decide what to do, I thought it would be OK to rely on the one difference we know there, the locale of the student. My reasoning was that Moodle does know about locales, and so we could safely rely on that.
However, your greater experience of teaching in a bilingual country suggests that we cannot always rely on the Moodle locale being set to the way that all students in the class will write numbers.
It is really a surprisingly difficult problem. We must not let this defeat us. There must be some brilliant way to solve this waiting to be discovered!
As things as not well structured here, I will put them in docs.
The analysis is done ( option 1,2,3) for the numbers that define the answers where we want all values with a .
i.e the teacher viewpoint either on edit_calculated_form or on import.
There is a somehow different analysis when handling the student response.
On the student response my main proposal is to add the decsep as option to answer as the tolerance parameter.
So a teacher could set two identical answers with different decsep and same grade or different grade as he want it.
We could then grade the decsep used directly and as we continue to use unit penalty , we are grading all 3 aspects on numerical value.
We could go further (in 2,2) and grade the thousandssep.
So the apply_unit used for answer will not be the same than the one used for the response.
We perhaps can continue to use the answer TEXT is we control the float conversion step so that it is always stored and retrieve with decsep = .
I am looking deep in the PHP source code and I think that we can control the decsep used in format() so we could store correctly.(so MDL27235 comment ).
PHP 5.3.6 string to double and is_numeric() use only . as decimal_point or decsep. ( to be tested ?)
It seems that we could handle correctly numbers using actual answer TEXT to store them.
 Do you agree to have an input answer unit_apply() function that will essentially help the teacher
to write correctly the answer numerical value following PHP syntax (no thousandssep and . as decsep).  To add a simple decsep choice as an additional answer parameter and some changes in the actual code as the decsep will no be controlled by lang files.
dec = ( , and .) i.e. we replace , by . with some rules
 only . no replacement
 only , as a first implementation for 2,1.
Handling the thousandssep variants is a more complex problem because we autorize exponential format so a student could eliminate them and some numbers don't need them (date i.e.).
We can just eliminate all no numbers char ( , space .) before the decsep so that the student response will be a strict PHP format with the exception of the decsep , being converted or not to . .
The grading need to be modified.
I can work the edit_numerical_form and the corresponding apply_value.
If you just sketch how you want the modifiction done on your apply_unit() I can work a proposal.
As a matter of fact storing the decsep in the attempt data is no more necessary.
Another advantage is that this renders numerical multilingual
Or ...
I think that:
1. On the editing form. The teacher should be able to type numbers using the teachers' locale, just like students can when attempting the quiz.
2. In the database, we should store numbers in PHP syntax.
Those two are clear to me.
I worry slightly about import and export. I think
3. For Moodle XML syntax, the numbers in the XML file should be PHP syntax, just like in the database. (So that a maths teacher in France can share maths questions with a teacher in England  but that does not really help with numbers typed into the question text. Hmm. Actually, in this scenario, I suppose the questiontext would be done using the multilang filter.)
4. For GIFT format, I am really not sure what is best. Teachers type GIFT format, so perhpas it should use localised numbers. On the other hand, if other import formats do something differente ...
Argh! It is all so complicated!
I agree,
I try to set some of the arguments about setting the answer field.
However, it was late so to continue tomorrow
The main conclusion is to train the teacher to use the PHP syntax to write a numerical quantity as Moodle code will always convert it in the answer field as a PHP valid number (as Excel is doing...).
However Moodle should help the teacher in this process.
When handling the student response, Moodle responsability is to grade the question with as much precision as possible for the 3 elements of the answer response
numerical quantity (tolerance)
unit
numerical syntax
Here the help should be the minimum necessary to do a fair grading.
This is why the apply_unit() should not be the same for answer and response...
Given the 2,1 deadline , I suggest that in 2,1 we maintain the 2,0 grading and display options of 2,0 numerical and calculated less the instructions as this is no more necessary.
As you have already set the code to display the multiple choice select option either at left we could maintain easily the number and unit two separate inputs.
You could then keep your improvments in the apply_unit() and the coding of decsep in attempt data for 2,2.
This will eliminate the bugs related to this issue and you can concentrate on other aspects of the new engine.
Just to remind you that the main features of the new engine for Moodle users are the new grading options.
Moodlers or Moodle don't ask you to develop in 2,1 all the possibilities of the new engine.
If you relax your constraints on internal code ( like rewriting all the calculated question type code to PHP 5 standards) for 2,1, you could polish all the grading options .
But effectively as you are a code engineer, you try to polish the internal engine although it is invisible to most Moodle users
This will allow me to develop my conceptions in a real proposal that we could polish before release to moodle community.
Some apsects f the other parallel project on questions could be integated in a 2,2 numerical and calculated interface.
The teacher should be able to set
1.the numerical value using the PHP convention
2.the response format , various interfaces can be used
3.the tolerance
4.the feedback
Here a simplified proposal (answer_resp.jpg
The answer response format parameters could be set individually or at the question level ( this disable individual answer response elements)
If we use these supplementary fields to the numerical answer
 decsep with (no , . and ,. ),
 thousandssep (any,, name )
 exponent formatting
 name (locale or moodle options name )
we could for 2,1 define three initial choices i.e.
 all formats as 2,0 ( use the 2,0 apply_unit)
 decsep, all compatible thousandssep ( use the new apply_unit)
 decsep , all compatible thousandssep ( use the new apply_unit)
The proposed fields should be sufficient for further developement in 2,2 ...
I have set a more detailed proposal on
http://docs.moodle.org/en/Development:Question_Engine_2:Numerical_formats#Grading_the_response_format
In testing 2,1 I just notice that (float) conversion stop on a non recognized character as letter other than e or E.
So if the answer number is 1234 then 1234, will be recognized as the number 1234 and the student will have full grade.
I will fix this in the new apply_unit() function.
Given the few days left, I am working on minimal changes that will allow 2,0 and strict , or . decsep to be used in 2,1 in such a way to be integrated easily in future more complex numerical response format gradings.
i.e. Defining for 2,1 three internal locales i.e. "decimal_point_.,", "decimal_point.","decimal_point_," will give enough info to use them in future version as we will need a locale parameter in future versions.
The less troublesome solution remains to use the 2,0 apply_unit(), don't store decsep parameters in the attempt data so that all new numerical grading will be implemented in 2,2.
This should be the case at least for calculated given the format parameters in calculated_answers.
Pierre, I have been following your comments here, and on Moodle docs. I am really glad you are thinking about this problem, because it is clearly hard, and I don't have time to address it properly.
I suspect that I will not have time to time to properly understand and review what you have done before the 31st May deadline for getting the the system basically finished and added to Moodle 2.1. However, I am sure it will be fine to incorporate your changes in the testing period between 31st May and 30th June. As you have said, it is important we get this right in 2.1, but I don't think we need to do it before 31st May.
So please come up with a good implementation of a good solution, so that when I come to look at it in early June, it will be easy for me You have now thought about this problem much more than I have, so I am sure you are in a better place to work out the right solution. However, please forgive me if I am not able to respond much to you over the next two weeks. Thanks.
As I told you somewhere, I always understand that you follow my work and as long as I am not in a bad issue, you have other priorities than to tell me its OK
More new tomorrow...( I am testing my hypothesis to use locale as an answer parameter.)
PROOF OF CONCEPT YAY..
Grading differently following dec sep.
Done using locale as additional answer parameter.
Need to have 2 inputs one for number and one for unit.
Three choices used in the prototype will be available in the first version
Moodle 2,0 flexible formats
decimal point .
decimal point ,
So back to sleep
Should we and how handle response data from preengine Moodle 1,9 attempts and 2,0?
For 2,0 Do you convert the "number  unit" format of 2,0 to a double entry ?
For 1,9 we should use the old apply_unit() to split the data ?
I can now grade all the combinations of units and numerical answer and apply the unit penalty.
I will leave for further work the implementation of numerical question_classified_responses to handle the case of
 good numerical and bad unit choice with applied unit penalty
 bad numerical value
 unrecognized numerical which is seen as a get_validation_error case ...
Improving the "simpletest" tests to the new design features.
So I am going back to the drawing board with the hypothesis to set the availabe number formats at the question level and necessary improvements of qtype_numerical_answer_processor.
The decsep, thousands and related parameters could then be saved part in the numerical_options table and or in a specific numerical/....php file.
first setting things in http://docs.moodle.org/en/Development:Question_Engine_2:Numerical_formats
As an example the new qtype_numerical_answer_processor could simulate the 2,0 apply_unit by setting the decsep by a simple analysis of the response and then setting the thousansseps with a multiple replacement of all related thousandseps to ''.
I any cases I strongly suggest that you do the coding
from MDL27410
'I really hope we can do that before 2.1 is released.'
let's keep it simplest.
As I concluded on http://docs.moodle.org/en/Development:Question_Engine_2:Numerical_formats#Improving_the_number_analysis_at_the_question_level
the decsep options should be set at the question level.
In future development we will propose a grading scenario that could apply successive number formats options (like units) with different grade.
So we will need something like

if ($oldversion < 2011052200 ) { //New version in version.php


/// Define table question_numerical_options to be created

$table = new xmldb_table('question_numerical_formats');


/// Adding fields to table question_numerical_formats

$table>add_field('id', XMLDB_TYPE_INTEGER, '10', XMLDB_UNSIGNED, XMLDB_NOTNULL, XMLDB_SEQUENCE, null);

$table>add_field('question', XMLDB_TYPE_INTEGER, '10', XMLDB_UNSIGNED, XMLDB_NOTNULL, null, '0');

$table>add_field('name', XMLDB_TYPE_TEXT, 'small', null, null, null, null);

$table>add_field('order', XMLDB_TYPE_INTEGER, '10', XMLDB_UNSIGNED, XMLDB_NOTNULL, null, '0');

$table>add_field('grade', XMLDB_TYPE_NUMBER, '12, 7', XMLDB_UNSIGNED, XMLDB_NOTNULL, null, '0');

$table>add_field('params', XMLDB_TYPE_TEXT, 'small', null, null, null, null);


/// Adding keys to table question_numerical_formats

$table>add_key('primary', XMLDB_KEY_PRIMARY, array('id'));

$table>add_key('question', XMLDB_KEY_FOREIGN, array('question'), 'question', array('id'));

/// Conditionally launch create table for question_calculated_options

if (!$dbman>table_exists($table)) {

// $dbman>create_table doesnt return a result, we just have to trust it

$dbman>create_table($table);

}

upgrade_plugin_savepoint(true, 2011052200, 'qtype', 'numerical');

}

to store the data.
for 2,1 we just propose 3 choices

class qtype_numerical_edit_form extends question_edit_form {


protected function definition_inner($mform) {


$localeoptions["dec_sep_,."]= "all" ;

$localeoptions["dec_sep_."]= "1234.56" ;

$localeoptions["dec_sep_,"]= "1234,56" ;

$mform>addElement('select', 'locale',

"Response format", $localeoptions);

for ["dec_sep_,."]= you either use the 2,0 apply_value() or you do some modifications to
qtype_numerical_answer_processor {
to detect which of , or . as the higher position in the response to switch from decsep , or .
In all cases as we want a "universal" decsep, the corresponding possible thousandseps should all be replaced by ''
i.e. decsep , thousandseps ' ' and .
/** @var array of string characters used as thousands separator . */
protected $thousandssep;
We should add the help icon to warn correctly the students.
This should do the minimal
You could maintain the unique input text and unit element.
For further developement we will see.
the 2,0 apply_unit() or its qtype_numerical_answer_processor version with the "dec_sep_,."]= "all" option should be used to validate the answers in the save_question_options ()
$answer>answer = $this>apply_unit($answerdata, $units,

!empty($question>unitsleft));

Upgrade should cope with upgrading attempts from 2.0 (and 1.9). The code is in question/type/numerical/db/upgradelib.php. You will see a class there with methods for updating the response data.
I will take a look tonight .
"However, I am sure it will be fine to incorporate your changes in the testing period between 31st May and 30th June. "
Is your proposal always valid as you have a lot of other things to do, I presume.
Or you have time to manage this and other aspects of grading units
I have some deadlines also although not serious as yours.
Sorting this out is one of the major loose ends before 30th June.
So I will prepare a proposal on this and unit grading for your return on Monday 6th June
As they are moving docs, I continue here
If we want to implement various numeric formats as
 Integer without decsep and with or without thousandseps (many combinations related to number locale) as 1234 1 234
 Decimal with a decsep ( either , or . ) and with or without thousandseps (many combinations related to number locale) as 1234.456 1 234,456
 Exponent form with a decsep ( either , or . ) and without thousandseps ? as 1.23456 E3
 Fractional form the dessep is ' ' and without thousandseps as 1234 456/1000
 Angles as degree minute second
 Time as hour:minute:second
 Etc.
We need to define different classes wich can have variants as  number with variants
 integer (no decsep)
 decimal decsep , or . and there corresponding thousanseps ( ' ' ',' '.')
 exponential format
 fractional format as 123 456/789
 time 13:30 or 10:30:23
 degrees 130^o^56'23"
We need to specify the order in the handling ( 1, 2, 3) and the grade % associated
if ($oldversion < 2011052200 ) { //New version in version.php


/// Define table question_numerical_options to be created

$table = new xmldb_table('question_numerical_formats');


/// Adding fields to table question_numerical_formats

$table>add_field('id', XMLDB_TYPE_INTEGER, '10', XMLDB_UNSIGNED, XMLDB_NOTNULL, XMLDB_SEQUENCE, null);

$table>add_field('question', XMLDB_TYPE_INTEGER, '10', XMLDB_UNSIGNED, XMLDB_NOTNULL, null, '0');

$table>add_field('order', XMLDB_TYPE_INTEGER, '10', XMLDB_UNSIGNED, XMLDB_NOTNULL, null, '0');

$table>add_field('grade', XMLDB_TYPE_NUMBER, '12, 7', XMLDB_UNSIGNED, XMLDB_NOTNULL, null, '0');

$table>add_field('variant', XMLDB_TYPE_TEXT, 'small', null, null, null, null);

$table>add_field('classname', XMLDB_TYPE_TEXT, 'small', null, null, null, null);

$table>add_field('params', XMLDB_TYPE_TEXT, 'small', null, null, null, null);


/// Adding keys to table question_numerical_formats

$table>add_key('primary', XMLDB_KEY_PRIMARY, array('id'));

$table>add_key('question', XMLDB_KEY_FOREIGN, array('question'), 'question', array('id'));

/// Conditionally launch create table for question_calculated_options

if (!$dbman>table_exists($table)) {

// $dbman>create_table doesnt return a result, we just have to trust it

$dbman>create_table($table);

}

upgrade_plugin_savepoint(true, 2011052200, 'qtype', 'numerical');

}

Given the time delay, let's just implement choosing the decsep for decimal with all the corresponding thousansseps.
So there will be 3 components to the grade
the value set by the answer
the % grade for good number syntax
the unit penalty
I will first create manually the table,
Create a first numeric_format class that will replace the new qtype_numerical_answer_processor (although using most of the code) as the decseps wil be set at the question level.
Implement a new grading process which imply testing the answer value with first the ordered number formats then the various units combinations. Will modify the unit tests accordngly.
On further thinking, we need another penalty for extranous characters in the number input response that could appear in the unit part of the decoding process even if the number is decoded correctly
array($beforepoint, $decimals, $exponent, $unit)
However if these extraneous characters define one of the available units and the unit field is empty then there should be no penalty but a simple warning to put unit in the unit input element (if not the multichoice option !)
So the simplest way to do it, is to apply to these extraneous characters the unit grading penalty.
A first proposal interface to allow to choose the decseps for a given question.
If the student does not use the prescribed decsep then he will receive a fraction of the grade corresponding to the numerical value of the response.
A better interface that allows to choose the descseps and the thousands seps.
Now let's do the internal...
This decsep problem is small part of the Question engine project.
What is more basic is that we don't set initially the coding so that further modifications will not be easy to do.
On the long term teachers should be able to grade number formats used in numerical ( decsep, integers, exponential format, fraction etc.)
The evolution from 1,6 to 2,0 has been to a more flexible handling of the various formats as there was no structure to a partial grading of the number formats.
I propose that for 2,1 we stay in this historical trend and apply progressively (even before 2,2) new features for number format analysis.
My trials on the number format grading show that it should be set at the question level (i.e. not a locale string neither as an answer property.
The proposed datatable (to which I have added decsep and thousandssep parameters)
numerical format stored
stdClass Object
(
[id] => 1
[question] => 27
[ordering] => 0
[grade] => 1.0000000
[variant] => general
[classname] => numerical
[params] =>
[decsep] => .
[thousandssep] => Array
(
[ ] =>
[,] => ,
)
)
can store the necessary parameters for many numerical variants necessary for number formats analysis.
decsep , or . define with their correspondinh thousandsseps most of the decimal variants.
A first simple grading schema is to apply first the first choice decsep and give 100% of the answer value is the response as translated with this decsep agree with the answer.
If not then we apply a second analysis using the other decsep and grade to 100% or 80% or ...
the answer value is the response as translated with this decsep agree with the answer.
This is the first simple two tests that I am working on.
See the last fig of the edit_numerical_form on this page.
I will post here tonight my last version.
However my recommandation is just to use in number analysis a more flexible process i.e do
first with decsep = . and thousandsseps = space or ,
then
with decsep = , and thousandsseps = space or .
and give the highest grade to the best one (i.e. 100% for the two decseps.)
For this, we only need to improve the apply_value() function.
Your actual proposal just handle one decsep and one thousandseps value.
We should also set back to the two separate inputs (number and unit) as we will need them to do the complex number analysis in the future.
After some supplemenary work, my proposal is the following:
for 2,1 using the http://tracker.moodle.org/secure/attachment/24021/grading%29numerical2.jpg interface
the teacher could define a prefered 100% grade decsepthousandsseps deimal configuration and give a var% to another one.
We need 2 numerical format records to do the minimal job as we need one numerical option record.
For actual question when they will be saved if the two records do not exist, they will be created automatically with the first one allowing decsep . and thousandsseps space and , 100% grade and the second one allowing decsep , and thousandsseps space and . 100% grade so giving all the actual language combination 100% (I need to study the half space proposal.
Furthermore for existing questions, when they will be used i.e. using qet_options, the two records could be created automatically (or at least contained the desired two data structures) so that the grading can be done with 2,1.
This could also be included in the migrating procedure.
If you agree with the proposal I can setup something in one day or 2.
Just a little work report
I can edit and save the numerical_format data and I have set the response input unit back and modified the question.php so that grading is working with either unit as separate text input or as multichoice.
Next thing to implement is the use of numerical_format data in the grading process.
I can decode a response using the numerical_format data with more one 1 thousandsseps in a step
i.e. decsep = . thousandsseps defined as array(' ',',') will decode 1 234.567 ands 1,234.67 to 1234.567
all within a "slightly" modified qtype_numerical_answer_processor.
So the main parts are OK. Let's assemble the "Meccano".
Grading reflects numerical_format grade
The numerical_format allows 70% of the answer>fraction to a number with decsep = ,
numerical formats


Array

(

[1] => stdClass Object

(

[id] => 1

[question] => 27

[ordering] => 0

[grade] => 1.0000000

[variant] => general

[classname] => numerical

[params] =>

[decsep] => .

[thousandssep] => Array

(

[ ] =>

[,] => ,

)


)


[2] => stdClass Object

(

[id] => 2

[question] => 27

[ordering] => 1

[grade] => 0.7000000

[variant] => general

[classname] => numerical

[params] =>

[decsep] => ,

[thousandssep] => Array

(

[ ] =>

[.] => .

)


)


)

the response was written as 1 815,5
and the answer was
Array

(

[32] => qtype_numerical_answer Object

(

[tolerance] => 0.1

[tolerancetype] => 2

[locale] =>

[id] => 32

[answer] => 1815.5

[answerformat] => 2

[fraction] => 1.0000000

[feedback] =>

[feedbackformat] => 1

)



after $this>get_matching_answer() but before grading the answer structure was
Array

(

[0] => qtype_numerical_answer Object

(

[tolerance] => 0.1

[tolerancetype] => 2

[locale] =>

[id] => 32

[answer] => 1815.5

[answerformat] => 2

[fraction] => 1.0000000

[feedback] =>

[feedbackformat] => 1

[unit] =>

[numberformatgrade] => 0.7000000

[value] => Array

(

[0] => 1815.5

[1] =>

)


)


)

So we have all infos to calculate the final grade.
$fraction = $answer>fraction * $answer>numberformatgrade;
if ($this>unitdisplay != qtype_numerical::UNITNONE )
Among other things, we will have to set how to handle these elements
in classify_response()
These last results show that the project could fulfill our needs for 2,1 and could be set to future needs.
I will continue to work on this draft proposal in the weekend and send you as it will be sunday night, so you can work on it next week.
Tim,
I just send you by email a rough proposal that handle numerical format grading and unit grading.
There are 3 situations where a parameter has to be analyzed for its content of a number part and possible unit part.
 in saving answers to extract the numerical value and convert it in a numerical PHP style ( 1234.567 or 1.23456e3)in a process that does not imply edit_numerical_form where an efficient validation is done. I propose to go back to the 2,0 apply_value code that could be improved by detecting which of , and . is the most at right so to conclude that it should be the decsep to use.
 This can be on import although in some cases as Gift the validation is already done
 This can be on upgrading the database. There is a notice that on old database the answer and the unit where saved in the same field. How old but if this apply to 2,1 ?
 In grading and the proposal handle correctly this case. However this is a more complex process than usual and it is used more than once in the rendering code. Could we store somewhere the first result?
 In the validating reponse where the analyze is somewhat different than in the grading process.
public function is_complete_response(array $response) {

if (!$this>is_gradable_response($response)) {

return false;

}


list($value, $unit) = $this>ap>apply_units($response['answer']);

if (is_null($value)) {

return false;

}

or

public function get_validation_error(array $response) {

if (!$this>is_gradable_response($response)) {

return get_string('pleaseenterananswer', 'qtype_numerical');

}


list($value, $unit) = $this>ap>apply_units($response['answer']);


the actual number written by the student could be 1 234.56 1,234.56 1 234,56 or 1.234,56
In the actual code using thousandsseps combination of 'space' ',' or '.' ',' will most often generate a number even if it is not the good one but can generate a apparent unit that renders the following tests non pertinent.
if (is_null($value)) {

return get_string('invalidnumber', 'qtype_numerical');

}



if ($this>unitdisplay != qtype_numerical::UNITINPUT && $unit) {

return get_string('invalidnumbernounit', 'qtype_numerical');

}


The solution could be to created another apply_units function or modify the actual.
I am not even convinced that these validation steps are the good ones as the final grade can be set to grade the 3 question parameters.
1 The number obtained from the applyunit form the allowed format is compare to the answer with all the combination obtained from the units using the tolerance interval. If Ok then the grade is the answer>value.
This is multiply by the numerical format grade (0100%)
Then the unit penalty can be applied.
Given this and various grading possibilites set for the question and the new options offered by the new engine, what is a complete response ?
I don't know the answer
Note this problem found by Eloy when testing another issue: http://tracker.moodle.org/browse/MDL27868?focusedCommentId=113420&page=com.atlassian.jira.plugin.system.issuetabpanels:commenttabpanel#comment113420
I have now worked out what I want to do with this: http://moodle.org/mod/forum/discuss.php?d=174387#p781440. Implementation will follow.
Note that, in addition to the plan in the forum post, the is_valid_answer function in the editing form needs to be fixed to use the same logic as save_question_options.
OK, I think I have implemented all of this.
Note that this branch follows on from the MDL27430 branch, which also relates to the numerical question type.
Finally note that you cannot currently test this in the question preview popup window because of MDL28074. I hope to fix that before submitting this for integration.
This is a somewhat scary change, if anyone can look at this, that would be great.
You are doing an almost "impossible mission" task...
Your last modifications are related to Students that could enter a thousand separator that is not a space char.
However your are using the locale decsep.
This is somewhat contradictory.
So let's play the game.
I am not part of the main moodle team, I can understand the deadlines are important to an organization as large as moodle.
However parts of the 2,1 will be more alpha than beta...
We could just expect that as summer in the northern hemisphere is mostly holiday period, few will migrate on 2,1 as official version so we could come back and polish all this numerical somehow complex issue.
I am aware it is somewhat contradictory to use thousands separator from the lang pack in the validation, but that correctly catches the situation where Moodle will misinterpret what the student meant, as I explained here: http://moodle.org/mod/forum/discuss.php?d=174387#p781440
My priority, before the Moodle release was to get Numerical to be mostly backwardscompatible, with some slight enhancements, and I think I have achieved that, but I am still happy for people to point out potential mistakes I may have made before it is too late.
I should add, that I looked at your proposed code (I got it onto github, see https://github.com/timhunt/moodle/tree/MDL27418_ppichet). I think that is a promising direction for Moodle 2.2. (or 2.1.x) but I have some concerns. Basically, while your changes add useful features to the question type, they also add a lot of complexity for people creating questions. I think for 90% of teachers, the added complexity is not worth it. So, I would like us to think about how we can make the good enhancements without increasing the complexity so much, if at all possible.

public function grade_response(array $response) {

if ($this>has_separate_unit_field()) {

$selectedunit = $response['unit'];

} else {

$selectedunit = null;

}

list($value, $unit) = $this>ap>apply_units($response['answer'], $selectedunit);

$answer = $this>get_matching_answer($value);

if (!$answer) {

return array(0, question_state::$gradedwrong);

}


$fraction = $this>apply_unit_penalty($answer>fraction, $unit);

return array($fraction, question_state::graded_state_for_fraction($fraction));

}

In the apply_units the $value from the response is divided by the unit multiplier if the unit is defined.
So lets take the following example (as I have only access to an Ipad I cannnot test code but I am using the last code on github.
So
the coorect answer is 123 cm and m has been defined so 1.23 m is the other good response.
the student type 123 cm everything is Ok
the student type 1.23 m everything is Ok
the student type 123 m then applyunit will convert this to 12300 and the grade will be 0 i.e. bad unit = 0% grade
samething with 1.23 cm
The process should be to test if the student value matches one of the possible number i.e. 123 or 1.23 then apply the unit penalty.
I put this here as I did not want to reopen the bug that is more related to this and already closed.
As far I as I can do on an Ipad , I look at your code and it is globally OK apart the last comment.
As for 2,2 we need to discuss more (I think we need a good chat exchange ).
The applyunit should not convert the value so that everything is handle outside as in my 2,2 proposal where I add also the various number decoding results.
Ah! I thought I had unit tests for the grading: https://github.com/timhunt/moodle/blob/MDL27418/question/type/numerical/simpletest/testquestion.php#L65
However I am missing your 123 m and 1.23 cm cases.
Actually, that test is doing the UNITOPTIONAL case. I need some new tests for UNITGRADED: https://github.com/timhunt/moodle/commit/c5aaf417390534ae1555a1826e12377597ffb8df
I can confirm your analysis is correct. Two of those new tests fail. Now, to make those tests pass!

/**

* Get an answer that contains the feedback and fraction that should be

* awarded for this resonse.

+ * test all units combination

+ * report the unit that gives the matching

+ * should test all decsep combination

+ * add the result to the corresponding answer ?

* @param number $value the numerical value of a response.

* @return question_answer the matching answer.

*/

 public function get_matching_answer($value) {

+ public function get_matching_answer($responseanswer, $selectedunit) {

foreach ($this>answers as $aid => $answer) {

 if ($answer>within_tolerance($value)) {

 $answer>id = $aid;

 return $answer;

+ foreach ($this>numericalformats as $key => $numericalformat) {

+ $this>ap>set_characters($numericalformat>decsep,$numericalformat>thousandssep); //,$selectedunit

+ list($value, $unit) = $this>ap>apply_units($responseanswer, $selectedunit,false);

+ if ($this>unitgradingtype) {

+ foreach( $this>ap>units as $keyunit=>$multiplier){

+ $test = $value/$multiplier ;

+ if ($answer>within_tolerance($test)) {

+ $answer>id = $aid;

+ $answer>unit = $keyunit;

+ $answer>selectedunit = $unit;

+ $answer>value = $value ;

+ $answer>numberformatgrade = $numericalformat>grade ;

+ $answerwithintolerances[]= $answer ;

+ }

+ }

+ }else{

+ if ($answer>within_tolerance($value)) {

+ $answer>id = $aid;

+ $answer>value = $value ;

+ $answer>numberformatgrade = $numericalformat>grade ;

+ $answerwithintolerances[]= $answer ;

+ }

+ }

}

 }

+ $maxkey = 0 ;

+ if(count($answerwithintolerances) > 0 ) {

+ $maxgrade = 0;

+ $maxkey = '' ;

+ foreach($answerwithintolerances as $key1 => $answerwithintolerance ){

+ if($answerwithintolerance>numberformatgrade > $maxgrade ){

+ $maxgrade = $answerwithintolerance>numberformatgrade ;

+ $maxkey = $key1 ;

+ }

+ }

+ }

+echo "<p>answer within_tolerance before maxgrade $maxgrade maxkey $maxkey<pre>";print_r($answerwithintolerances);echo "</pre></p>";

+ // return which as the greatest $answer>numberformatgrade

+ return $answerwithintolerances[$maxkey];

+ }

+

return null;

 }

+ }


if you eliminate the foreach related to the numberformat, and adapt for the applyunit, you have the main structure to solve the problem.
However I assume that you have already a similar way to solve the problem.
If not then then i will be happy to leanr a new way to do it.
About your concerns for 2,1.x complex interface one solution will be to allow more than one advance field in moodle form.
we could then have a simple interface with just answer,
or number foormat and answer
or answer and unit
or number format answer and unit
but less finish 2,1 first

+ foreach( $this>ap>units as $keyunit=>$multiplier){

+ $test = $value/$multiplier ;

+ if ($answer>within_tolerance($test)) {

+ $answer>id = $aid;

+ $answer>unit = $keyunit;

+ $answer>selectedunit = $unit;

+ $answer>value = $value ;

+ $answer>numberformatgrade = $numericalformat>grade ;

+ $answerwithintolerances[]= $answer ;

+ }

+ }

+ }else{

+ if ($answer>within_tolerance($value)) {

+ $answer>id = $aid;

+ $answer>value = $value ;

+ $answer>numberformatgrade = $numericalformat>grade ;

+ $answerwithintolerances[]= $answer ;

+ }

+ }

}

This was missing (just in case) and Ipad does not allow reediting among other things...
I have been agonising all day about what 'Right number, but wrong unit' really means.
I don't like the 2.0 definition which involves multiplying all possible numerical answers by all possible unit multipliers. I think that is too hard to understand and for the teacher to control.
In the end, I decided 'right numerical value' meant just the right number typed.
So, if the question has right answer 1.23 m, with second unit 100 cm = 1 m, then:
Right: 1.23 m or 123 cm
Right number, wrong unit: 1.23 or 1.23 frogs or 1.23 cm
Wrong: 123 m
If the teacher wants the response 123 m to be graded partially correct, then they must add 123 as a second answer.
This may not be 100% right, but I think it correctly grades almost all cases I can think of. At least this is the best I can do before 2.1 is released. Fingers crossed.
I somehow begin to understand why we don't have the same point of view about what is unit penalty ?
If I come back to my real classroom experience, when I ask student to calculate say the surface of a cercle, I expect a numerical figure and a unit in accordance with the numerical unit and somewhere on their exam I should see a pi* r2 calculation.
Most important did they put the correct r (if for example I gave them the diameter).
So the good numerical can be different if they use different unit ( we are more than bilingual in québec, we are multiunits ) cm, m inches etc...
If they have one of these values, then they have a good numerical response.
Then and only then I can apply the penalty unit if they get loss in the unit expressed. I have even different unit penalties in a case as area (if the unit used is an area unit the penalty is less than if they use a linear unit ...)
This is what I try to expressed in all my docs (and in the related code), but it seems that I failed as a teacher
Your experience as a computer expert could be related to your way to see things this way. If you program say a figure dimension and you put a size too large, it won't fit in the screen and the result is bad...
You could be reluctant to give to such a result a good grade ???
On the other end, this could be corrected in the first release days.
As a final note, this is a minor part of all the great work you have done...
P.S. If you can do it, put it back as I suggest and if everything is not OK, I can easily take the blame at my age
"If the teacher want 123 m to be graded partially correct he should set 123 as a partial graded answer"
This could be the case if the units were defiend differently as without a first unit with a unity multiplication faactor and the other ones with specific multiplication factors.
In this initial logic where the multiply factor was associated with a given unit, if the student does not use the good unit which has not 1 as multiplier then the grade was 0.
In a unit penalty schema the unit is not if it is not written correctly or not the good unit or even is not given. Discriminating between these bad units will be much more complex.
What I want to explain is that when we set a unit and its multiplier with a value different from one,we set that the resulting number i.e 123 is as valid as 1.23.
So it should grade as 1.23
I am trying to imagine your scenario, and I find it very difficult to visualise in a way that is convincing. What I imagine is a question like:
"A circle has radius 5cm, what is its area? (remember to give the correct unit)"
Accepted units cm^2, m^2, inch^2
Correct answer: 78.5398163 cm^2
I do not believe that a student would ever try to give the response using any unit other than cm^2 after reading that question text.
Of course we should be able to verify this when you get back from holiday. Looking in the DB to see what responses your students really gave for some real questions.
I expect I am imagining the wrong sort of example question. Please continue to teach me.
I also think that since ancient history, the unit handling in the numerical qtype has been really terrible. What you did in 2.0 was a really clever way to make it much more useful, while still being mostly backwardscompatible. However, being backwards compatible is really hard.
I wonder if the best longterm plan would be:
1. Keep qtype_numeric for numberonly questions  deprecate all the unit options there.
2. Make a completely new qtype, for example qtype_numberandunit, where we can implement unit grading in a logical way, without having to worry about backwardscompatibility.
Integrated, yay!
(finally added one more commit about singleline whitespace fix)
Thanks for all the hard brainstorming Pierre and Tim!
"I expect that I am imagining the wrong sort of example question. Please continue to teach me"
Effectively in such a question, I will never imagine a teacher setting the various units and the coefficient on a moodle question
However just change your question by
A circle has a radius of 0,00005km, what is its area ?(remember to give the correct unit)
Accepted units are cm2 or m2 ( I will not ask for inch2 as this imply another conversion.
In such a case part of the students will first convert to m so radius is 0,05m and will either
calculate with 0,05 m
or convert this to 5 cm which is easier to calculate.
This is why the unit option was set with the multiplier in Moodle.
As far as I remember, there is no multiply factor in WebCT although there is a unit penalty factor.So the unit penalty is more simple to code.
And this multiplier is the main reason why we should grade by testing all the units number combination. It effectively sets different numerical values that can test against the correct answer value.
We don't need to add another 123 value as you suggest upward.
That is a more convincing example. Since this issue is now closed, I have created MDL28118 so we don't forget about this issue.
Got error while creating numerical question.
Notice: Undefined variable: x in /usr/local/www/moodle/question/type/numerical/edit_numerical_form.php on line 294 Call Stack: 0.0014 879608 1.
This is happening because of change in question/type/numerical/edit_numerical_form.php#294 (in function is_valid_answer).
Just a note that MDL28138 was created to deal with the problem in the previous comment. I am looking at it.
Hello Tim,
There are two functions which seems to be broken.
One I mentioned earlier (is_valid_answer), is a bloker.
Other mentioned in MDL28138 (float_format) is not a blocker, but yeah it will be nice to have a look at it.
Hope this helps.
Cheers
Rajesh
Oh, I did not realise about the is_valid_ansewr thing.
Looking.
Meanwhile, MDL28138 is ready for integration.
let me grrr myself a bit, I missed those completely
Fixed, with an extra commit https://github.com/timhunt/moodle/commit/e42646b167e10462abe025d8781ddf2fe0364961 on the end of the MDL27418 branch.
Please integrate, and sorry for my mistake.
no worries, Tim! Reintegrated, thanks!
Works Great
Thanks for providing the patch Tim.
Upstreamized! Thanks!
The apply_unit() is used
1.in saving the answers
2.in grading the student responses.
The 2,2 apply_unit() converts number to the PHP internal convention i.e. thousandssep (, ,space) are eliminated and decsep , is converted to . if it is the only one and that there is a . used.
the newengine apply_unit() eliminates the thousandssep associated with the thousandssep and converts the decsep associate to the actual user (either teacher or student) language .
What are the needs
1.in saving the answers
We need an universal conversion of written numbers to the internal PHP number to help the teacher set correctly the answer in PHP internal number format.
We shoudl add help to the editing form and get a more explicit is_numeric() validate message.
actual being
$string['invalidnumericanswer'] = 'One of the answers you entered was not a valid number.';
$string['invalidnumerictolerance'] = 'One of the tolerances you entered was not a valid number.';
For import we should use actual 2,2 that should be improved
i.e detecting all variants of thousandssep (i.e. half space)
or detecting the position of the , and .
etc.
2. In handling the response a 2,2 style should be the default for 2,0 and preceeding version (i.e. imported ) questions because
we don't know if the teacher have given or not indications on the number writing.
or as we will do at UQAM migrate from a french site 1.9 with . convention learned by 40 000 students to a 2 version with a , mandatory ?
So we need at least a specific field to identify at least the decsep which was used when creating a question.
It should be a the answer level.
Given that decsep can be , or . and that thousandssep possible values are limited to not many values, the choice offered to the teacher could be more a list of the available combination than a complete list of all the defined locales.
I will experiment on this.
I have set a working branch from qe2_wip and will start to work first on the edit and import problem using a specific 2,0 style apply_unit() function.
The following from PHP.net http://ca2.php.net/manual/en/class.numberformatter.php
illustrates well the problem
Also from unix as PHP docs refer to UNIX http://linux.about.com/library/cmd/blcmdl3_strtod.htm
We cannot take for granted that the PHP locale settings are constant on moodle installations ...
Is this could be a problem when saving answer as in
MDL27325?