Moodle
  1. Moodle
  2. MDL-12433

get_string support for plurals and some other grammar via simple conditional expressions

    Details

    • Type: Improvement Improvement
    • Status: Open
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Affects Version/s: 2.0
    • Fix Version/s: None
    • Component/s: Libraries
    • Labels:
      None
    • Rank:
      4000

      Description

      get_string should support plurals in a manner that works for multiple languages and is backward-compatible.

      This solution is based on Petr's idea. The idea is that it uses the same language string 'normally', but if you give a parameter which is a low number, it looks for an alternate version of the string that ends in _1 (or _0 or _2 or _3).

      Here are some example language strings:

      Original string:
      $string['blah'] = '$a green bottles sitting on the wall';

      Plural strings:
      $string['blah_1'] = 'One green bottle sitting on the wall';
      $string['blah_2'] = 'Two green bottles sitting on the wall';

      Code using this:
      print_object(get_string('blah','mymodule',1)); // One green bottle sitting on the wall
      print_object(get_string('blah','mymodule',2)); // Two green bottles sitting on the wall
      print_object(get_string('blah','mymodule',3)); // 3 green bottles sitting on the wall

      Some notes about the logic here:

      1. This code supports only numbers 0, 1, 2, and 3 for this special handling. In English there are cases where special handling is appropriate for 0 ('none', different wording) and 1 (singular). According to http://en.wikipedia.org/wiki/Grammatical_number#Dual_number there are some other languages which need different handling for n=2 (and some archaic ones that need different handling for n=3).

      2. The search route is completely unchanged. This code only operates after it has already found the base string ('blah' in the case above). It only ever finds plurals specified in the same place.

      This is necessary because plurals might be implemented in one language but not another. For example if there is a blah_1 for English, but only blah for French, the user should still see the French version in preference. This does have the consequence that it is not possible to override a plural language string without also overriding the singular version.

      3. Since get_string is frequently called, I have written the code in such a way as to maximise performance. I am unable to find a measurable before/after difference in performance (variation between tests is greater than the difference). [Note that our performance tests used a PHP accelerator; we presume that heavily-loaded sites will also be in this situation.]

      See attached patch for code.

      1. get_string_expressions.patch
        7 kB
        Sam Marshall
      2. get_string_plurals.patch
        6 kB
        Sam Marshall
      3. translationchange.patch
        30 kB
        Sam Marshall
      1. new-language-editor-screen.png
        31 kB

        Issue Links

          Activity

          Hide
          Sam Marshall added a comment -

          I made a forum post about this: http://moodle.org/mod/forum/discuss.php?d=85897

          Show
          Sam Marshall added a comment - I made a forum post about this: http://moodle.org/mod/forum/discuss.php?d=85897
          Hide
          David Mudrak added a comment -

          In some languages - and they do not need to be archaic ones - the rules are more complicated.
          See http://moodle.org/mod/forum/discuss.php?d=58516 for more general solution.

          Show
          David Mudrak added a comment - In some languages - and they do not need to be archaic ones - the rules are more complicated. See http://moodle.org/mod/forum/discuss.php?d=58516 for more general solution.
          Hide
          Sam Marshall added a comment -

          Note that there is more discussion in that forum thread and I no longer think this particular solution is a good idea (which is a shame, since I coded it - code first, ask questions later, that's me).

          However if we can come to an agreement on something in the thread, I will update this bug to propose that.

          Show
          Sam Marshall added a comment - Note that there is more discussion in that forum thread and I no longer think this particular solution is a good idea (which is a shame, since I coded it - code first, ask questions later, that's me). However if we can come to an agreement on something in the thread, I will update this bug to propose that.
          Hide
          Sam Marshall added a comment -

          Here (patch get_string_expressions) is a revised effort to solve the issue based on the discussion in the forum. This obsoletes the older patch, and solves a somewhat wider range of problems.

          The patch does two things:

          1) Allows $string19[..]=... in language files. This syntax will do nothing in 1.8 but (after this patch) will work the same as $string[...] in 1.9. This means you can override a string in 1.9.

          Note that this behaviour should be used only for cases where a new language file feature needs to be used - not when the actual code (e.g. parameters) changes. In those cases the string name should be changed, as at present.

          2) Allows arrays to be used as a string value. These are associative arrays with the key being a PHP expression and the value being a language string in existing format. The array is evaluated in order and the first item for which the PHP expression returns true (or an item where the expression is '') will be selected as the language string to use.

          Here is an example language string using the new features:

          $string['bottles']='$a green bottles';
          $string19['bottles']=array(
          '$a==1'=>'one green bottle',
          ''=>'$a green bottles');

          Before the patch, Moodle will display '$a green bottles'. After the path, it will display either 'one green bottle' (if the parameter is 1) or '$a green bottles' (if it's something else).

          As you can see it is easy to support dual languages or include more complex rules. (At a later point it would be desirable to allow languages to define their own functions, which would be able to support more complex decisions via this format.)

          I have tested the performance of this change, before and after, when reading standard language strings with and without parameters. (But not using the new features - i.e. this is a 'did it break anything?' test.) Average time taken for these get_string calls on a PHP 5.1x test server with a PHP accelerator is as follows:

          Before patch: 0.062ms
          After patch: 0.066ms (5% slower than current)
          After patch and MDL-12434 performance change: 0.059ms (10% faster than current)

          This patch does not include changes to language translation UI tools that may be necessary.

          Show
          Sam Marshall added a comment - Here (patch get_string_expressions) is a revised effort to solve the issue based on the discussion in the forum. This obsoletes the older patch, and solves a somewhat wider range of problems. The patch does two things: 1) Allows $string19 [..] =... in language files. This syntax will do nothing in 1.8 but (after this patch) will work the same as $string [...] in 1.9. This means you can override a string in 1.9. Note that this behaviour should be used only for cases where a new language file feature needs to be used - not when the actual code (e.g. parameters) changes. In those cases the string name should be changed, as at present. 2) Allows arrays to be used as a string value. These are associative arrays with the key being a PHP expression and the value being a language string in existing format. The array is evaluated in order and the first item for which the PHP expression returns true (or an item where the expression is '') will be selected as the language string to use. Here is an example language string using the new features: $string ['bottles'] ='$a green bottles'; $string19 ['bottles'] =array( '$a==1'=>'one green bottle', ''=>'$a green bottles'); Before the patch, Moodle will display '$a green bottles'. After the path, it will display either 'one green bottle' (if the parameter is 1) or '$a green bottles' (if it's something else). As you can see it is easy to support dual languages or include more complex rules. (At a later point it would be desirable to allow languages to define their own functions, which would be able to support more complex decisions via this format.) I have tested the performance of this change, before and after, when reading standard language strings with and without parameters. (But not using the new features - i.e. this is a 'did it break anything?' test.) Average time taken for these get_string calls on a PHP 5.1x test server with a PHP accelerator is as follows: Before patch: 0.062ms After patch: 0.066ms (5% slower than current) After patch and MDL-12434 performance change: 0.059ms (10% faster than current) This patch does not include changes to language translation UI tools that may be necessary.
          Hide
          Sam Marshall added a comment -

          Correction: 'after patch and performance change' was 5% faster than current, not 10%.

          Show
          Sam Marshall added a comment - Correction: 'after patch and performance change' was 5% faster than current, not 10%.
          Hide
          Sam Marshall added a comment -

          Here is a new all-singing all-dancing patch. This patch applies to HEAD (and 1.9 incidentally, but as noted already, this is too big a change to make 1.9 now) and works with language files as follows:

          $string['normparam'] = '$a parameters';
          $except['normparam'] = array(
          '$a==7'=>'$a is the lucky number!');

          So $string is the 'normal' values; $except may contain an array of condition => value.

          You cannot have $except unless you also have $string. The two together are treated as a single unit i.e. if you override the string, but don't override the except, then your 'except' is now blank. [This is on purpose - these rules should make it work sensibly.]

          Most notably this patch includes full support (I think) for the translation editor. Incidentally, that code is really horrific! It's all in one giant file... sigh.

          I will attach a screenshot next.

          Note that this patch might have some rough edges (aka BUGS) because I only just finished it right now. I am not intending to do any more work to it unless this is accepted for inclusion in core; if it is, I guess I'll check it in (in the new year) then accept bugs filed on it. After this week I am away for a couple weeks so won't do anything until I get back from that.

          Show
          Sam Marshall added a comment - Here is a new all-singing all-dancing patch. This patch applies to HEAD (and 1.9 incidentally, but as noted already, this is too big a change to make 1.9 now) and works with language files as follows: $string ['normparam'] = '$a parameters'; $except ['normparam'] = array( '$a==7'=>'$a is the lucky number!'); So $string is the 'normal' values; $except may contain an array of condition => value. You cannot have $except unless you also have $string. The two together are treated as a single unit i.e. if you override the string, but don't override the except, then your 'except' is now blank. [This is on purpose - these rules should make it work sensibly.] Most notably this patch includes full support (I think) for the translation editor. Incidentally, that code is really horrific! It's all in one giant file... sigh. I will attach a screenshot next. Note that this patch might have some rough edges (aka BUGS) because I only just finished it right now. I am not intending to do any more work to it unless this is accepted for inclusion in core; if it is, I guess I'll check it in (in the new year) then accept bugs filed on it. After this week I am away for a couple weeks so won't do anything until I get back from that.
          Hide
          Sam Marshall added a comment -

          Screenshot of the language editor showing all features. Should be pretty clear how this works.

          Show
          Sam Marshall added a comment - Screenshot of the language editor showing all features. Should be pretty clear how this works.
          Hide
          Sam Marshall added a comment -

          SORRY for doing like a million updates to this bug today, but this is the last one I promise (I'm not supposed to stay in the building after 10 minutes ago!)

          1) It is a bit unclear, the current patch is 'translationchange.patch', the much bigger one. This replaces the earlier patches, which are now obsolete.

          2) I started a new forum thread about this issue to reflect that I actually have a complete patch that includes the editor. http://moodle.org/mod/forum/discuss.php?d=86912

          Show
          Sam Marshall added a comment - SORRY for doing like a million updates to this bug today, but this is the last one I promise (I'm not supposed to stay in the building after 10 minutes ago!) 1) It is a bit unclear, the current patch is 'translationchange.patch', the much bigger one. This replaces the earlier patches, which are now obsolete. 2) I started a new forum thread about this issue to reflect that I actually have a complete patch that includes the editor. http://moodle.org/mod/forum/discuss.php?d=86912
          Hide
          Eloy Lafuente (stronk7) added a comment -

          Hi,

          IMO this is enough mature to have a real chance to go to HEAD, anyway I would propose to:

          1) Do some performance tests with complex pages having, say, 100 exceptions.
          2) Allow one last iteration of the idea and implementation (explained to non-technical translators) to see if we are missing something important or the implementation needs something else.

          Just my opinion, ciao

          Show
          Eloy Lafuente (stronk7) added a comment - Hi, IMO this is enough mature to have a real chance to go to HEAD, anyway I would propose to: 1) Do some performance tests with complex pages having, say, 100 exceptions. 2) Allow one last iteration of the idea and implementation (explained to non-technical translators) to see if we are missing something important or the implementation needs something else. Just my opinion, ciao
          Hide
          Eloy Lafuente (stronk7) added a comment -

          Also, one more point... I'm a bit reticent about adding php code directly in lang strings (the conditions). Could we consider defining them as formulas:

          parameter > 4
          parameter == 10
          parameter->field = 5

          and measure performance under that approach? Just guessing if it's easy to do.... :-/

          Ciao

          Show
          Eloy Lafuente (stronk7) added a comment - Also, one more point... I'm a bit reticent about adding php code directly in lang strings (the conditions). Could we consider defining them as formulas: parameter > 4 parameter == 10 parameter->field = 5 and measure performance under that approach? Just guessing if it's easy to do.... :-/ Ciao
          Hide
          Koen Roggemans added a comment -

          I am worried about that too. For inspiration to get worried, see http://tracker.moodle.org/browse/MDL-14152

          Show
          Koen Roggemans added a comment - I am worried about that too. For inspiration to get worried, see http://tracker.moodle.org/browse/MDL-14152
          Hide
          Helen Foster added a comment -

          Just noting that the issue of incorrect pluralization was mentioned in the CANnect Accessibility report by Randall Hansen - see linked issue MDL-20430.

          Show
          Helen Foster added a comment - Just noting that the issue of incorrect pluralization was mentioned in the CANnect Accessibility report by Randall Hansen - see linked issue MDL-20430 .
          Hide
          Sam Marshall added a comment -

          This issue was assigned to me automatically, however I will not be able to work on this issue in the immediate future. In order to create a truer sense of the state of this issue and to allow other developers to have chance to become involved, I am removing myself as the assignee of this issue.

          For more information, see http://docs.moodle.org/dev/Changes_to_issue_assignment

          Show
          Sam Marshall added a comment - This issue was assigned to me automatically, however I will not be able to work on this issue in the immediate future. In order to create a truer sense of the state of this issue and to allow other developers to have chance to become involved, I am removing myself as the assignee of this issue. For more information, see http://docs.moodle.org/dev/Changes_to_issue_assignment

            People

            • Votes:
              6 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated: