Moodle
  1. Moodle
  2. MDL-15252

META Clean-up and reorganisation of language packs in Moodle 2.0

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.0
    • Fix Version/s: 2.0.3
    • Component/s: Language
    • Labels:
    • Affected Branches:
      MOODLE_20_STABLE
    • Fixed Branches:
      MOODLE_20_STABLE
    • Rank:
      355

      Description

      Since Moodle 2.0 is going to be a major release, I suggest a break in the backward compatibility of the English language pack (wich is not backwards compatible anymore anyway).
      All strings, not used in Moodle 2.0 should be removed, so the initial translation for a new language pack is less work and less threathening then the + 10k strings there are now.

      The other language packs will keep on being backward compatible due to the orphaned function in lang.php

        Issue Links

          Activity

          Hide
          Koen Roggemans added a comment -

          Thinking a little bit more on this: there is a problem with the help files: deleting them or moving them around will break backward compatibility.

          May be we need to redo the trick from 1.6 over again: create a new folder in CVS: lang2, clean all packs and move them over, have a new download page for 2.x language packs

          Show
          Koen Roggemans added a comment - Thinking a little bit more on this: there is a problem with the help files: deleting them or moving them around will break backward compatibility. May be we need to redo the trick from 1.6 over again: create a new folder in CVS: lang2, clean all packs and move them over, have a new download page for 2.x language packs
          Hide
          Nicolas Martignoni added a comment -

          I would be glad if language packs maintainers for other languages have the possibility to do the same cleanup for 2.0.

          My +1 for the idea in Koen's comment.

          Show
          Nicolas Martignoni added a comment - I would be glad if language packs maintainers for other languages have the possibility to do the same cleanup for 2.0. My +1 for the idea in Koen's comment.
          Hide
          David Mudrak added a comment -

          My +1 for Koen's idea. It should be enough to clean up the English pack. As Koen noticed above, already translated strings will remain in other lang packs marked as // ORPHANED.
          It should not be hard to write a script that will search the the whole Moodle source tree looking for get_string() and print_string() call and extracting the list of currently used strings. I can participate on writing such a report.

          Show
          David Mudrak added a comment - My +1 for Koen's idea. It should be enough to clean up the English pack. As Koen noticed above, already translated strings will remain in other lang packs marked as // ORPHANED. It should not be hard to write a script that will search the the whole Moodle source tree looking for get_string() and print_string() call and extracting the list of currently used strings. I can participate on writing such a report.
          Hide
          Martin Dougiamas added a comment -

          I'd be interested in this, if only to see how many strings are actually not in use. Wonder if we should think about removing some of the duplicates too ..?

          Show
          Martin Dougiamas added a comment - I'd be interested in this, if only to see how many strings are actually not in use. Wonder if we should think about removing some of the duplicates too ..?
          Hide
          Eloy Lafuente (stronk7) added a comment -

          Hi,

          just trying to revive this a bit... I've created the MDL-16769 subtask.

          It's about to create one script to seach for unused strings across moodle/contrib given a lang pack.

          David, I've initially assigned it to you. In case that isn't ok, please reassign it to me.

          I think that such script can give us some interesting numbers to see, definetivelly, what to do for 2.0 (a good chance to break everything, as you know).

          Ciao

          Show
          Eloy Lafuente (stronk7) added a comment - Hi, just trying to revive this a bit... I've created the MDL-16769 subtask. It's about to create one script to seach for unused strings across moodle/contrib given a lang pack. David, I've initially assigned it to you. In case that isn't ok, please reassign it to me. I think that such script can give us some interesting numbers to see, definetivelly, what to do for 2.0 (a good chance to break everything, as you know). Ciao
          Hide
          Helen Foster added a comment -

          Thinking again about this issue whilst going through the grades help files. It seems very confusing having help files for the 1.9 gradebook in the same folder as help files for the pre-1.9 gradebook.

          From chatting with Eloy:

          stronk7: well, they are two different things afaik:

          1) cleanup of unused string/help files
          2) branching the rest of languages

          1 requires 2 i.e. we cannot perform the cleanup properly without branching, but branching langs has it risks too:

          1) translators must LEARN to use and commit them properly (potential nightmare)
          2) translation interface must support branches
          3) lang download (both from web page and from within moodle must know about branches)

          (that's the summary IMO) but, for sure, for any cleanup, we need branching

          Helen: or as Koen says: Maybe we need to redo the trick from 1.6 over again: create a new folder in CVS: lang2, clean all packs and move them over, have a new download page for 2.x language packs

          stronk7: yes, that's an option... but in the long term (2.3, 2.4...) we'll be in the same situation.
          but perhaps "lang2" is a good moment, and we could start with it branched

          stronk7: perhaps one discussion in forums could help?

          asking translators and so on (they are the "key" to see if branching is going to be fairly managed by them)
          they'll need to have one Moodle 1.9 another Moodle 2.0, Moodle 2.1 and perform each translation in the corresponding onw (committing to corresponding branch).

          Show
          Helen Foster added a comment - Thinking again about this issue whilst going through the grades help files. It seems very confusing having help files for the 1.9 gradebook in the same folder as help files for the pre-1.9 gradebook. From chatting with Eloy: stronk7: well, they are two different things afaik: 1) cleanup of unused string/help files 2) branching the rest of languages 1 requires 2 i.e. we cannot perform the cleanup properly without branching, but branching langs has it risks too: 1) translators must LEARN to use and commit them properly (potential nightmare) 2) translation interface must support branches 3) lang download (both from web page and from within moodle must know about branches) (that's the summary IMO) but, for sure, for any cleanup, we need branching Helen: or as Koen says: Maybe we need to redo the trick from 1.6 over again: create a new folder in CVS: lang2, clean all packs and move them over, have a new download page for 2.x language packs stronk7: yes, that's an option... but in the long term (2.3, 2.4...) we'll be in the same situation. but perhaps "lang2" is a good moment, and we could start with it branched stronk7: perhaps one discussion in forums could help? asking translators and so on (they are the "key" to see if branching is going to be fairly managed by them) they'll need to have one Moodle 1.9 another Moodle 2.0, Moodle 2.1 and perform each translation in the corresponding onw (committing to corresponding branch).
          Hide
          Koen Roggemans added a comment -

          In a productive traffic jam I came up with a 3th option that might address some difficulties.
          We could install on a Moodle box all maintained Moodle versions, give translators an admin account on it with the translating capability set. A demo course can be stored (and may be restored every X time) to have something to play with. The box updates daily
          A robot can daily move the translated files in the correct CVS branch.

          This solves
          -no need to do anything with CVS for translators - branching is automatically
          -no need to install anything for translators: translating is done on the Moodle servers
          -no need to change anything on the translation interface

          This is problematic AFAIK for
          -moodle.com: yet another set of installations to maintain
          -translators who like playing with CVS
          -translators who like translating directly on the files.
          Maybe those last two groups can have a CVS account to do their thing on the files, but I would keep that as an exeptions

          Show
          Koen Roggemans added a comment - In a productive traffic jam I came up with a 3th option that might address some difficulties. We could install on a Moodle box all maintained Moodle versions, give translators an admin account on it with the translating capability set. A demo course can be stored (and may be restored every X time) to have something to play with. The box updates daily A robot can daily move the translated files in the correct CVS branch. This solves -no need to do anything with CVS for translators - branching is automatically -no need to install anything for translators: translating is done on the Moodle servers -no need to change anything on the translation interface This is problematic AFAIK for -moodle.com: yet another set of installations to maintain -translators who like playing with CVS -translators who like translating directly on the files. Maybe those last two groups can have a CVS account to do their thing on the files, but I would keep that as an exeptions
          Hide
          Martin Dougiamas added a comment -

          Good idea Koen. Might help collaboration in some instances too.

          Jordan can look after the technical side of running these sites and robots etc, but people familiar with translations are going to need to lead the cleanup effort (of course this can be done from any CVS checkout).

          Show
          Martin Dougiamas added a comment - Good idea Koen. Might help collaboration in some instances too. Jordan can look after the technical side of running these sites and robots etc, but people familiar with translations are going to need to lead the cleanup effort (of course this can be done from any CVS checkout).
          Hide
          Koen Roggemans added a comment -

          We need a clean Moodle 2.0 language pack in English. Then the orphaned function should go out of the translation interface. Each language file that is edited after that, will be clean.
          The help files is a manual job.
          Is there something else you think off?

          Show
          Koen Roggemans added a comment - We need a clean Moodle 2.0 language pack in English. Then the orphaned function should go out of the translation interface. Each language file that is edited after that, will be clean. The help files is a manual job. Is there something else you think off?
          Hide
          Nicolas Martignoni added a comment -

          +1 for Koen's idea. Advantages are that with such a solution we can cope with translators with technical background AND with CVS-aware ones.

          It could enhance too the cooperation/collaboration in translation teams.

          Show
          Nicolas Martignoni added a comment - +1 for Koen's idea. Advantages are that with such a solution we can cope with translators with technical background AND with CVS-aware ones. It could enhance too the cooperation/collaboration in translation teams.
          Hide
          Mitsuhiro Yoshida added a comment -

          I like Koen's idea too

          So, if we start translating collaboratively, how can we avoid conflict of translation, ideas and political stuff?
          And who can manage translation groups for each language?.

          Show
          Mitsuhiro Yoshida added a comment - I like Koen's idea too So, if we start translating collaboratively, how can we avoid conflict of translation, ideas and political stuff? And who can manage translation groups for each language?.
          Hide
          Koen Roggemans added a comment -

          That is what I occasionally deal with behind the screens. I am a little bit worried for editing wars, but not everyone can edit the language pack - there is always human interaction necessary to give extra people the right to edit.

          At the moment, all translations that are sent to translation@moodle.org go to the language pack maintainer with CVS access. Otherwise it is send to the one that did the last send, if that is a reasonable time ago (say less then a year ago).

          We'll have to come up with new strategies for language packs with occasional translation efforts ...

          Show
          Koen Roggemans added a comment - That is what I occasionally deal with behind the screens. I am a little bit worried for editing wars, but not everyone can edit the language pack - there is always human interaction necessary to give extra people the right to edit. At the moment, all translations that are sent to translation@moodle.org go to the language pack maintainer with CVS access. Otherwise it is send to the one that did the last send, if that is a reasonable time ago (say less then a year ago). We'll have to come up with new strategies for language packs with occasional translation efforts ...
          Hide
          Mitsuhiro Yoshida added a comment -

          > That is what I occasionally deal with behind the screens.

          Yes, I have experienced some in 6 years

          Show
          Mitsuhiro Yoshida added a comment - > That is what I occasionally deal with behind the screens. Yes, I have experienced some in 6 years
          Hide
          David Mudrak added a comment -

          Hello world, some quick notes to this... (copy goes to the forum as well)

          Helen has mentioned some issues with the help files for the new gradebook. The current correct way is to introduce new helpfiles (eg. foobar2.html instead of current foobar.html) instead of changing the current ones so the backward compatibility is preserved. But we know this leads to a lot of obsoleted strings and helpfiles...

          I would really love to see language packages being branched. I think we will have to help our translators to use CVS branching features. More documentation, step-by-step tutorials etc. is needed but after several sucessful commits this might become an easy daily task. How many translators use CVS at the moment?

          Eloy noted that the translation tool itself should support branches. How? The translation is done against the English pack at a given branch. How would we get English files from several branches into a single installation?

          Ad collaborative translation: to be honest, I am very sceptical here. IMHO every language pack needs a maintainer (Pythonists use term BDFL) who makes decisions. Code maintaining is about communication, not permission. I am little bit afraid of terminology wars, mentioned by Koen.

          Let me be brave and propose using GIT instead of CVS for language packs (what??? is David crazy???). Not only branching and merging is much easier in GIT than in CVS. But it also fits our needs and process much better. Try to imagine:

          • every language pack is maintained by a single "official" translator. They keep branches for all stable releases.
          • anybody can decide to create their own fork (clone) of the official language pack repository and start making modifications, changes etc.
          • the maintainer can decide to pull changes from other repositories, compare them against their own etc.
          • language ZIP packs are generated automagically from all repositories available.
          • every site administrator may choose which version (clone) they want to use.
          Show
          David Mudrak added a comment - Hello world, some quick notes to this... (copy goes to the forum as well) Helen has mentioned some issues with the help files for the new gradebook. The current correct way is to introduce new helpfiles (eg. foobar2.html instead of current foobar.html) instead of changing the current ones so the backward compatibility is preserved. But we know this leads to a lot of obsoleted strings and helpfiles... I would really love to see language packages being branched. I think we will have to help our translators to use CVS branching features. More documentation, step-by-step tutorials etc. is needed but after several sucessful commits this might become an easy daily task. How many translators use CVS at the moment? Eloy noted that the translation tool itself should support branches. How? The translation is done against the English pack at a given branch. How would we get English files from several branches into a single installation? Ad collaborative translation: to be honest, I am very sceptical here. IMHO every language pack needs a maintainer (Pythonists use term BDFL) who makes decisions. Code maintaining is about communication, not permission. I am little bit afraid of terminology wars, mentioned by Koen. Let me be brave and propose using GIT instead of CVS for language packs (what??? is David crazy???). Not only branching and merging is much easier in GIT than in CVS. But it also fits our needs and process much better. Try to imagine: every language pack is maintained by a single "official" translator. They keep branches for all stable releases. anybody can decide to create their own fork (clone) of the official language pack repository and start making modifications, changes etc. the maintainer can decide to pull changes from other repositories, compare them against their own etc. language ZIP packs are generated automagically from all repositories available. every site administrator may choose which version (clone) they want to use.
          Hide
          Eloy Lafuente (stronk7) added a comment -

          > Eloy noted that the translation tool itself should support branches. How? The translation is done against the English pack at a given branch. How would we get English files from several branches into a single installation?

          Agree, correct, translation tool only will know about current installed lang files. It's the download part the one that must know about new lang18/lang19/lang20/lang21/lang directories (and the web page in downloads.moodle.org to show all them).

          >Ad collaborative translation: to be honest, I am very sceptical here.

          Disagree, I think that offering one Translation site for each branch is a good solution and a lot of translators can be using it online instead of local installations. Those wanting to use own servers + direct access to CVS can continue using that. Only difference is they'll need to handle branches themselves.

          >Let me be brave and propose using GIT instead of CVS for language packs

          • 100% disagree (for langs in particular and for moodle in general). People wanting to have clones can use git or whatever. But we need the centric approach of CVS. And merging between branches isn't really a pain if you know what is being done.

          In any case, perhaps we could have one 1...10 list of tasks needed to fulfil this change? I think it's mandatory to have all the steps clearly agreed (mainly because it affects CURRENT 1.9 lang packs) before doing anything. One page in the Docs could be a good place to make this evolve IMO (apart of discussion having place here/in forums).

          Ciao

          Show
          Eloy Lafuente (stronk7) added a comment - > Eloy noted that the translation tool itself should support branches. How? The translation is done against the English pack at a given branch. How would we get English files from several branches into a single installation? Agree, correct, translation tool only will know about current installed lang files. It's the download part the one that must know about new lang18/lang19/lang20/lang21/lang directories (and the web page in downloads.moodle.org to show all them). >Ad collaborative translation: to be honest, I am very sceptical here. Disagree, I think that offering one Translation site for each branch is a good solution and a lot of translators can be using it online instead of local installations. Those wanting to use own servers + direct access to CVS can continue using that. Only difference is they'll need to handle branches themselves. >Let me be brave and propose using GIT instead of CVS for language packs 100% disagree (for langs in particular and for moodle in general). People wanting to have clones can use git or whatever. But we need the centric approach of CVS. And merging between branches isn't really a pain if you know what is being done. In any case, perhaps we could have one 1...10 list of tasks needed to fulfil this change? I think it's mandatory to have all the steps clearly agreed (mainly because it affects CURRENT 1.9 lang packs) before doing anything. One page in the Docs could be a good place to make this evolve IMO (apart of discussion having place here/in forums). Ciao
          Hide
          David Mudrak added a comment -

          > offering one Translation site for each branch is a good solution

          as I just answered in the forum, the reason of my worries is the number of users that can actually save the strings and help files. If we open such possibility to anybody coming and saying "hey I want to make this language pack better" then editing wars may come (maybe, maybe not - wikipedia works pretty well ). If this centralized hosting is meant just as a service to the current language pack maintainers who want it, I've no problem with it.

          > -100% git

          ok, np. It was just a brainstorming idea. You are right: if someone wants to actually use git mirrors, they already can do so. Although I still think that switching to git would be good step for Moodle project, there is no need to start with us, poor translators :-p

          Show
          David Mudrak added a comment - > offering one Translation site for each branch is a good solution as I just answered in the forum, the reason of my worries is the number of users that can actually save the strings and help files. If we open such possibility to anybody coming and saying "hey I want to make this language pack better" then editing wars may come (maybe, maybe not - wikipedia works pretty well ). If this centralized hosting is meant just as a service to the current language pack maintainers who want it, I've no problem with it. > -100% git ok, np. It was just a brainstorming idea. You are right: if someone wants to actually use git mirrors, they already can do so. Although I still think that switching to git would be good step for Moodle project, there is no need to start with us, poor translators :-p
          Hide
          Dan Poltawski added a comment -

          (This is going a bit off topic, so sorry)

          > -100% git

          I agree it shouldn't be introduced to translations, but this discussion did make me consider an extension to Koens idea which i'd write down - you could have moodle commit to a local git repository on the translation server (with the author credentials of the moodle user) as a way of tracking changes which happen in the translation between automated commits. If we were feeling really adventurous this could also be used as a way for any moodle user to make translations and then get reviewed for commit.

          Show
          Dan Poltawski added a comment - (This is going a bit off topic, so sorry) > -100% git I agree it shouldn't be introduced to translations, but this discussion did make me consider an extension to Koens idea which i'd write down - you could have moodle commit to a local git repository on the translation server (with the author credentials of the moodle user) as a way of tracking changes which happen in the translation between automated commits. If we were feeling really adventurous this could also be used as a way for any moodle user to make translations and then get reviewed for commit.
          Hide
          David Mudrak added a comment -

          Rewording the issue title as it evolved into a META umbrella.

          Show
          David Mudrak added a comment - Rewording the issue title as it evolved into a META umbrella.
          Hide
          David Mudrak added a comment -

          All the clean-up should be done more or less easily in the new AMOS framework.

          Show
          David Mudrak added a comment - All the clean-up should be done more or less easily in the new AMOS framework.
          Hide
          Tim Hunt added a comment -

          David, just reading the "Re-committing all English strings exported from AMOS ... " Commit comment. Particularly the "Where there was no copyright note so far, I added the
          default one with Martin Dougiamas as the copyright holder." bit.

          Surely Amos knows who first committed each lang file, and which year, and so could generate correct copyright notices?

          Show
          Tim Hunt added a comment - David, just reading the "Re-committing all English strings exported from AMOS ... " Commit comment. Particularly the "Where there was no copyright note so far, I added the default one with Martin Dougiamas as the copyright holder." bit. Surely Amos knows who first committed each lang file, and which year, and so could generate correct copyright notices?
          Hide
          David Mudrak added a comment -

          Thanks for the idea Tim. Unfortunately, for reasonable reasons, AMOS tracks the strings not from the beginning but from the time we switched to UTF8 for Moodle 1.6. Therefore, Martin is the first committer of all English string files and Eloy is the first committer of all UTF8 translations. To make things even more complicated, Eloy committed the migrated translations before Martin moved the English originals. So, in AMOS history, the initial translations are born before their parents...

          I am sure we will be able to regenerate the copyright later. Right now, I do not consider it priority. I actually followed the precedence - IIRC we have added Martin as the default copyright holder for all files that did not state their holder explicitly.

          Show
          David Mudrak added a comment - Thanks for the idea Tim. Unfortunately, for reasonable reasons, AMOS tracks the strings not from the beginning but from the time we switched to UTF8 for Moodle 1.6. Therefore, Martin is the first committer of all English string files and Eloy is the first committer of all UTF8 translations. To make things even more complicated, Eloy committed the migrated translations before Martin moved the English originals. So, in AMOS history, the initial translations are born before their parents... I am sure we will be able to regenerate the copyright later. Right now, I do not consider it priority. I actually followed the precedence - IIRC we have added Martin as the default copyright holder for all files that did not state their holder explicitly.
          Hide
          David Mudrak added a comment -

          All subtasks resolved, closing META.

          Show
          David Mudrak added a comment - All subtasks resolved, closing META.
          Hide
          Helen Foster added a comment -

          Great work David, many thanks!

          Show
          Helen Foster added a comment - Great work David, many thanks!

            People

            • Votes:
              2 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: