Moodle

get_string support for plurals and some other grammar via simple conditional expressions

Details

  • Type: Improvement Improvement
  • Status: Open Open
  • Priority: Minor Minor
  • Resolution: Unresolved
  • Affects Version/s: 2.0
  • Fix Version/s: None
  • Component/s: Libraries
  • Labels:
    None
  • Affected Branches:
    MOODLE_20_STABLE

Description

get_string should support plurals in a manner that works for multiple languages and is backward-compatible.

This solution is based on Petr's idea. The idea is that it uses the same language string 'normally', but if you give a parameter which is a low number, it looks for an alternate version of the string that ends in _1 (or _0 or _2 or _3).

Here are some example language strings:

Original string:
$string['blah'] = '$a green bottles sitting on the wall';

Plural strings:
$string['blah_1'] = 'One green bottle sitting on the wall';
$string['blah_2'] = 'Two green bottles sitting on the wall';

Code using this:
print_object(get_string('blah','mymodule',1)); // One green bottle sitting on the wall
print_object(get_string('blah','mymodule',2)); // Two green bottles sitting on the wall
print_object(get_string('blah','mymodule',3)); // 3 green bottles sitting on the wall

Some notes about the logic here:

1. This code supports only numbers 0, 1, 2, and 3 for this special handling. In English there are cases where special handling is appropriate for 0 ('none', different wording) and 1 (singular). According to http://en.wikipedia.org/wiki/Grammatical_number#Dual_number there are some other languages which need different handling for n=2 (and some archaic ones that need different handling for n=3).

2. The search route is completely unchanged. This code only operates after it has already found the base string ('blah' in the case above). It only ever finds plurals specified in the same place.

This is necessary because plurals might be implemented in one language but not another. For example if there is a blah_1 for English, but only blah for French, the user should still see the French version in preference. This does have the consequence that it is not possible to override a plural language string without also overriding the singular version.

3. Since get_string is frequently called, I have written the code in such a way as to maximise performance. I am unable to find a measurable before/after difference in performance (variation between tests is greater than the difference). [Note that our performance tests used a PHP accelerator; we presume that heavily-loaded sites will also be in this situation.]

See attached patch for code.

Attachments

  1. Text File
    get_string_expressions.patch
    07/Dec/07 12:02 AM
    7 kB
    Sam Marshall
  2. Text File
    get_string_plurals.patch
    04/Dec/07 10:26 PM
    6 kB
    Sam Marshall
  3. Text File
    translationchange.patch
    20/Dec/07 3:47 AM
    30 kB
    Sam Marshall
  1. new-language-editor-screen.png
    31 kB
    20/Dec/07 3:50 AM

Issue Links

This issue blocks:
MDL-12417 Database 'insufficient entries' string is not very good Trivial Open
This issue will help resolve:
MDL-20430 Pluralization is often incorrect Minor Open
This issue will be resolved by:
MDL-4790 Language-specific grammar doesn't exist Trivial Open

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Sam Marshall added a comment -

I made a forum post about this: http://moodle.org/mod/forum/discuss.php?d=85897

Show
Sam Marshall added a comment - I made a forum post about this: http://moodle.org/mod/forum/discuss.php?d=85897
Hide
Permalink
David Mudrak added a comment -

In some languages - and they do not need to be archaic ones - the rules are more complicated.
See http://moodle.org/mod/forum/discuss.php?d=58516 for more general solution.

Show
David Mudrak added a comment - In some languages - and they do not need to be archaic ones - the rules are more complicated. See http://moodle.org/mod/forum/discuss.php?d=58516 for more general solution.
Hide
Permalink
Sam Marshall added a comment -

Note that there is more discussion in that forum thread and I no longer think this particular solution is a good idea (which is a shame, since I coded it - code first, ask questions later, that's me).

However if we can come to an agreement on something in the thread, I will update this bug to propose that.

Show
Sam Marshall added a comment - Note that there is more discussion in that forum thread and I no longer think this particular solution is a good idea (which is a shame, since I coded it - code first, ask questions later, that's me). However if we can come to an agreement on something in the thread, I will update this bug to propose that.
Hide
Permalink
Sam Marshall added a comment -

Here (patch get_string_expressions) is a revised effort to solve the issue based on the discussion in the forum. This obsoletes the older patch, and solves a somewhat wider range of problems.

The patch does two things:

1) Allows $string19[..]=... in language files. This syntax will do nothing in 1.8 but (after this patch) will work the same as $string[...] in 1.9. This means you can override a string in 1.9.

Note that this behaviour should be used only for cases where a new language file feature needs to be used - not when the actual code (e.g. parameters) changes. In those cases the string name should be changed, as at present.

2) Allows arrays to be used as a string value. These are associative arrays with the key being a PHP expression and the value being a language string in existing format. The array is evaluated in order and the first item for which the PHP expression returns true (or an item where the expression is '') will be selected as the language string to use.

Here is an example language string using the new features:

$string['bottles']='$a green bottles';
$string19['bottles']=array(
'$a==1'=>'one green bottle',
''=>'$a green bottles');

Before the patch, Moodle will display '$a green bottles'. After the path, it will display either 'one green bottle' (if the parameter is 1) or '$a green bottles' (if it's something else).

As you can see it is easy to support dual languages or include more complex rules. (At a later point it would be desirable to allow languages to define their own functions, which would be able to support more complex decisions via this format.)

I have tested the performance of this change, before and after, when reading standard language strings with and without parameters. (But not using the new features - i.e. this is a 'did it break anything?' test.) Average time taken for these get_string calls on a PHP 5.1x test server with a PHP accelerator is as follows:

Before patch: 0.062ms
After patch: 0.066ms (5% slower than current)
After patch and MDL-12434 performance change: 0.059ms (10% faster than current)

This patch does not include changes to language translation UI tools that may be necessary.

Show
Sam Marshall added a comment - Here (patch get_string_expressions) is a revised effort to solve the issue based on the discussion in the forum. This obsoletes the older patch, and solves a somewhat wider range of problems. The patch does two things: 1) Allows $string19[..]=... in language files. This syntax will do nothing in 1.8 but (after this patch) will work the same as $string[...] in 1.9. This means you can override a string in 1.9. Note that this behaviour should be used only for cases where a new language file feature needs to be used - not when the actual code (e.g. parameters) changes. In those cases the string name should be changed, as at present. 2) Allows arrays to be used as a string value. These are associative arrays with the key being a PHP expression and the value being a language string in existing format. The array is evaluated in order and the first item for which the PHP expression returns true (or an item where the expression is '') will be selected as the language string to use. Here is an example language string using the new features: $string['bottles']='$a green bottles'; $string19['bottles']=array( '$a==1'=>'one green bottle', ''=>'$a green bottles'); Before the patch, Moodle will display '$a green bottles'. After the path, it will display either 'one green bottle' (if the parameter is 1) or '$a green bottles' (if it's something else). As you can see it is easy to support dual languages or include more complex rules. (At a later point it would be desirable to allow languages to define their own functions, which would be able to support more complex decisions via this format.) I have tested the performance of this change, before and after, when reading standard language strings with and without parameters. (But not using the new features - i.e. this is a 'did it break anything?' test.) Average time taken for these get_string calls on a PHP 5.1x test server with a PHP accelerator is as follows: Before patch: 0.062ms After patch: 0.066ms (5% slower than current) After patch and MDL-12434 performance change: 0.059ms (10% faster than current) This patch does not include changes to language translation UI tools that may be necessary.
Hide
Permalink
Sam Marshall added a comment -

Correction: 'after patch and performance change' was 5% faster than current, not 10%.

Show
Sam Marshall added a comment - Correction: 'after patch and performance change' was 5% faster than current, not 10%.
Hide
Permalink
Sam Marshall added a comment -

Here is a new all-singing all-dancing patch. This patch applies to HEAD (and 1.9 incidentally, but as noted already, this is too big a change to make 1.9 now) and works with language files as follows:

$string['normparam'] = '$a parameters';
$except['normparam'] = array(
'$a==7'=>'$a is the lucky number!');

So $string is the 'normal' values; $except may contain an array of condition => value.

You cannot have $except unless you also have $string. The two together are treated as a single unit i.e. if you override the string, but don't override the except, then your 'except' is now blank. [This is on purpose - these rules should make it work sensibly.]

Most notably this patch includes full support (I think) for the translation editor. Incidentally, that code is really horrific! It's all in one giant file... sigh.

I will attach a screenshot next.

Note that this patch might have some rough edges (aka BUGS) because I only just finished it right now. I am not intending to do any more work to it unless this is accepted for inclusion in core; if it is, I guess I'll check it in (in the new year) then accept bugs filed on it. After this week I am away for a couple weeks so won't do anything until I get back from that.

Show
Sam Marshall added a comment - Here is a new all-singing all-dancing patch. This patch applies to HEAD (and 1.9 incidentally, but as noted already, this is too big a change to make 1.9 now) and works with language files as follows: $string['normparam'] = '$a parameters'; $except['normparam'] = array( '$a==7'=>'$a is the lucky number!'); So $string is the 'normal' values; $except may contain an array of condition => value. You cannot have $except unless you also have $string. The two together are treated as a single unit i.e. if you override the string, but don't override the except, then your 'except' is now blank. [This is on purpose - these rules should make it work sensibly.] Most notably this patch includes full support (I think) for the translation editor. Incidentally, that code is really horrific! It's all in one giant file... sigh. I will attach a screenshot next. Note that this patch might have some rough edges (aka BUGS) because I only just finished it right now. I am not intending to do any more work to it unless this is accepted for inclusion in core; if it is, I guess I'll check it in (in the new year) then accept bugs filed on it. After this week I am away for a couple weeks so won't do anything until I get back from that.
Hide
Permalink
Sam Marshall added a comment -

Screenshot of the language editor showing all features. Should be pretty clear how this works.

Show
Sam Marshall added a comment - Screenshot of the language editor showing all features. Should be pretty clear how this works.
Hide
Permalink
Sam Marshall added a comment -

SORRY for doing like a million updates to this bug today, but this is the last one I promise (I'm not supposed to stay in the building after 10 minutes ago!)

1) It is a bit unclear, the current patch is 'translationchange.patch', the much bigger one. This replaces the earlier patches, which are now obsolete.

2) I started a new forum thread about this issue to reflect that I actually have a complete patch that includes the editor. http://moodle.org/mod/forum/discuss.php?d=86912

Show
Sam Marshall added a comment - SORRY for doing like a million updates to this bug today, but this is the last one I promise (I'm not supposed to stay in the building after 10 minutes ago!) 1) It is a bit unclear, the current patch is 'translationchange.patch', the much bigger one. This replaces the earlier patches, which are now obsolete. 2) I started a new forum thread about this issue to reflect that I actually have a complete patch that includes the editor. http://moodle.org/mod/forum/discuss.php?d=86912
Hide
Permalink
Eloy Lafuente (stronk7) added a comment -

Hi,

IMO this is enough mature to have a real chance to go to HEAD, anyway I would propose to:

1) Do some performance tests with complex pages having, say, 100 exceptions.
2) Allow one last iteration of the idea and implementation (explained to non-technical translators) to see if we are missing something important or the implementation needs something else.

Just my opinion, ciao

Show
Eloy Lafuente (stronk7) added a comment - Hi, IMO this is enough mature to have a real chance to go to HEAD, anyway I would propose to: 1) Do some performance tests with complex pages having, say, 100 exceptions. 2) Allow one last iteration of the idea and implementation (explained to non-technical translators) to see if we are missing something important or the implementation needs something else. Just my opinion, ciao
Hide
Permalink
Eloy Lafuente (stronk7) added a comment -

Also, one more point... I'm a bit reticent about adding php code directly in lang strings (the conditions). Could we consider defining them as formulas:

parameter > 4
parameter == 10
parameter->field = 5

and measure performance under that approach? Just guessing if it's easy to do.... :-/

Ciao

Show
Eloy Lafuente (stronk7) added a comment - Also, one more point... I'm a bit reticent about adding php code directly in lang strings (the conditions). Could we consider defining them as formulas: parameter > 4 parameter == 10 parameter->field = 5 and measure performance under that approach? Just guessing if it's easy to do.... :-/ Ciao
Hide
Permalink
Koen Roggemans added a comment -

I am worried about that too. For inspiration to get worried, see http://tracker.moodle.org/browse/MDL-14152

Show
Koen Roggemans added a comment - I am worried about that too. For inspiration to get worried, see http://tracker.moodle.org/browse/MDL-14152
Hide
Permalink
Helen Foster added a comment -

Just noting that the issue of incorrect pluralization was mentioned in the CANnect Accessibility report by Randall Hansen - see linked issue MDL-20430.

Show
Helen Foster added a comment - Just noting that the issue of incorrect pluralization was mentioned in the CANnect Accessibility report by Randall Hansen - see linked issue MDL-20430.

People

Vote (5)
Watch (8)

Dates

  • Created:
    Updated:
Atlassian JIRA (v4.4.3#663-r165197) | Report a problem
Powered by a free Atlassian JIRA open source license for Moodle. Try JIRA - bug tracking software for your team.