Moodle
  1. Moodle
  2. MDL-7993

Problems with Unicode generated Excel spreadsheets...

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.6.3, 1.7
    • Fix Version/s: 1.6.5, 1.7.2, 1.8
    • Component/s: Gradebook
    • Labels:
      None
    • Environment:
      Generating any XLS sheet
    • Affected Branches:
      MOODLE_16_STABLE, MOODLE_17_STABLE
    • Fixed Branches:
      MOODLE_16_STABLE, MOODLE_17_STABLE, MOODLE_18_STABLE
    • Rank:
      27611

      Description

      In order to be Unicode compliant, we are using one "special" mode of the PEAR excel library that allows to write UTF-16LE, so we can convert utf8=>utf16 and write them properly.

      The problem is that such "special" mode is some unestable for some big sheets (XLS limits AFAIK). I've done tons of tests and sheets are broken randomly, depending of the contents.

      The proposal is (there are two):

      1) Allow one admin setting to go back to the "default" mode (iso-8859-1). It will help to all the iso-8859-1 compliant langs (PEAR excel library rocks under iso). Could fit easily inside 1.7.1 and above.
      2) Build one "OpenDocument" alternative. I've been doing one small research across Internet and it seems that there isn't any product ready yet. Perhaps we could build it (shouldn't be really difficult in theory, It's XML) and help the whole PHPers Community! (1.8...)

      If approved, I would split this into two subtasks.

      Ciao

        Issue Links

          Activity

          Hide
          Eloy Lafuente (stronk7) added a comment -

          assigning to MD!

          Show
          Eloy Lafuente (stronk7) added a comment - assigning to MD!
          Hide
          Petr Škoda added a comment -

          here is a little xmas present - basic OpenDocument spreadsheet export class with similar interface as our Excel export

          Show
          Petr Škoda added a comment - here is a little xmas present - basic OpenDocument spreadsheet export class with similar interface as our Excel export
          Hide
          Martin Dougiamas added a comment -

          Cool!

          Show
          Martin Dougiamas added a comment - Cool!
          Hide
          Eloy Lafuente (stronk7) added a comment -

          Cool, Petr.

          Are you going to handle 1) too (option to generate iso-8859-1 sheets under Excel based on admin setting).

          Show
          Eloy Lafuente (stronk7) added a comment - Cool, Petr. Are you going to handle 1) too (option to generate iso-8859-1 sheets under Excel based on admin setting).
          Hide
          Petr Škoda added a comment -

          3/ add Open XML export option - MS compatibility pack for Office 2000, XP and 2003 allows opening/saving of Open XML documents http://office.microsoft.com/en-us/products/HA101686761033.aspx

          I am not sure what to change to implement 1

          Show
          Petr Škoda added a comment - 3/ add Open XML export option - MS compatibility pack for Office 2000, XP and 2003 allows opening/saving of Open XML documents http://office.microsoft.com/en-us/products/HA101686761033.aspx I am not sure what to change to implement 1
          Hide
          Eloy Lafuente (stronk7) added a comment -

          Hi Petr,

          great job all the OpenDocument stuff!

          Related with 1) I've sent to HEAD the modified lib/excelib.class.php file that, based on the $CFG->excelisofiles (true/false) value, will generate iso-8859-1 files that are stronger than the default utf-16 ones. I know that this setting is only useful for iso-8859-1 people but, well, they are a lot

          TODO:

          a) Add $CFG->excelisofiles somewhere in the admin interface.
          b) Backport the changes to 17_STABLE (in real world I've tested my changes against two moodle servers running 1.7 and results were 100% successful with some sheets not being properly generated.

          It would be great if you can do it. I'll be out the next 10 days (27th to 4th)....

          Ciao

          Show
          Eloy Lafuente (stronk7) added a comment - Hi Petr, great job all the OpenDocument stuff! Related with 1) I've sent to HEAD the modified lib/excelib.class.php file that, based on the $CFG->excelisofiles (true/false) value, will generate iso-8859-1 files that are stronger than the default utf-16 ones. I know that this setting is only useful for iso-8859-1 people but, well, they are a lot TODO: a) Add $CFG->excelisofiles somewhere in the admin interface. b) Backport the changes to 17_STABLE (in real world I've tested my changes against two moodle servers running 1.7 and results were 100% successful with some sheets not being properly generated. It would be great if you can do it. I'll be out the next 10 days (27th to 4th).... Ciao
          Hide
          Petr Škoda added a comment -

          OK, no problem - I will implement it

          Show
          Petr Škoda added a comment - OK, no problem - I will implement it
          Hide
          Eloy Lafuente (stronk7) added a comment -

          Oki, I'm back-porting the library to 1.6 and 1.7 and adding the config interface to 1.8 and HEAD.

          Show
          Eloy Lafuente (stronk7) added a comment - Oki, I'm back-porting the library to 1.6 and 1.7 and adding the config interface to 1.8 and HEAD.
          Hide
          Eloy Lafuente (stronk7) added a comment -

          Updated excel library is now in 16_STABLE, 17_STABLE, 18_STABLE and HEAD. Now it encodes to windows-1252 because if adds support for some more characters not present in iso-8859-1.

          Also, although working, for correctness, the $CFG->excelisofiles setting although working have been deprecated in favour of $CFG->latinexcelexport.

          Next step: Admin interface for this setting 18_STABLE and HEAD.

          Show
          Eloy Lafuente (stronk7) added a comment - Updated excel library is now in 16_STABLE, 17_STABLE, 18_STABLE and HEAD. Now it encodes to windows-1252 because if adds support for some more characters not present in iso-8859-1. Also, although working, for correctness, the $CFG->excelisofiles setting although working have been deprecated in favour of $CFG->latinexcelexport. Next step: Admin interface for this setting 18_STABLE and HEAD.
          Hide
          Eloy Lafuente (stronk7) added a comment -

          Subtask MDL-8585 created for HQ in order to decide where to implement the setting....

          Show
          Eloy Lafuente (stronk7) added a comment - Subtask MDL-8585 created for HQ in order to decide where to implement the setting....
          Hide
          Martin Dougiamas added a comment -

          Looks great! Marking as fixed.

          Show
          Martin Dougiamas added a comment - Looks great! Marking as fixed.

            People

            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: