Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-7993

Problems with Unicode generated Excel spreadsheets...

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.6.3, 1.7
    • Fix Version/s: 1.6.5, 1.7.2, 1.8
    • Component/s: Gradebook
    • Labels:
      None
    • Environment:
      Generating any XLS sheet
    • Affected Branches:
      MOODLE_16_STABLE, MOODLE_17_STABLE
    • Fixed Branches:
      MOODLE_16_STABLE, MOODLE_17_STABLE, MOODLE_18_STABLE

      Description

      In order to be Unicode compliant, we are using one "special" mode of the PEAR excel library that allows to write UTF-16LE, so we can convert utf8=>utf16 and write them properly.

      The problem is that such "special" mode is some unestable for some big sheets (XLS limits AFAIK). I've done tons of tests and sheets are broken randomly, depending of the contents.

      The proposal is (there are two):

      1) Allow one admin setting to go back to the "default" mode (iso-8859-1). It will help to all the iso-8859-1 compliant langs (PEAR excel library rocks under iso). Could fit easily inside 1.7.1 and above.
      2) Build one "OpenDocument" alternative. I've been doing one small research across Internet and it seems that there isn't any product ready yet. Perhaps we could build it (shouldn't be really difficult in theory, It's XML) and help the whole PHPers Community! (1.8...)

      If approved, I would split this into two subtasks.

      Ciao

        Gliffy Diagrams

          Issue Links

            Activity

            Hide
            stronk7 Eloy Lafuente (stronk7) added a comment -

            assigning to MD!

            Show
            stronk7 Eloy Lafuente (stronk7) added a comment - assigning to MD!
            Hide
            skodak Petr Skoda added a comment -

            here is a little xmas present - basic OpenDocument spreadsheet export class with similar interface as our Excel export

            Show
            skodak Petr Skoda added a comment - here is a little xmas present - basic OpenDocument spreadsheet export class with similar interface as our Excel export
            Hide
            dougiamas Martin Dougiamas added a comment -

            Cool!

            Show
            dougiamas Martin Dougiamas added a comment - Cool!
            Hide
            stronk7 Eloy Lafuente (stronk7) added a comment -

            Cool, Petr.

            Are you going to handle 1) too (option to generate iso-8859-1 sheets under Excel based on admin setting).

            Show
            stronk7 Eloy Lafuente (stronk7) added a comment - Cool, Petr. Are you going to handle 1) too (option to generate iso-8859-1 sheets under Excel based on admin setting).
            Hide
            skodak Petr Skoda added a comment -

            3/ add Open XML export option - MS compatibility pack for Office 2000, XP and 2003 allows opening/saving of Open XML documents http://office.microsoft.com/en-us/products/HA101686761033.aspx

            I am not sure what to change to implement 1

            Show
            skodak Petr Skoda added a comment - 3/ add Open XML export option - MS compatibility pack for Office 2000, XP and 2003 allows opening/saving of Open XML documents http://office.microsoft.com/en-us/products/HA101686761033.aspx I am not sure what to change to implement 1
            Hide
            stronk7 Eloy Lafuente (stronk7) added a comment -

            Hi Petr,

            great job all the OpenDocument stuff!

            Related with 1) I've sent to HEAD the modified lib/excelib.class.php file that, based on the $CFG->excelisofiles (true/false) value, will generate iso-8859-1 files that are stronger than the default utf-16 ones. I know that this setting is only useful for iso-8859-1 people but, well, they are a lot

            TODO:

            a) Add $CFG->excelisofiles somewhere in the admin interface.
            b) Backport the changes to 17_STABLE (in real world I've tested my changes against two moodle servers running 1.7 and results were 100% successful with some sheets not being properly generated.

            It would be great if you can do it. I'll be out the next 10 days (27th to 4th)....

            Ciao

            Show
            stronk7 Eloy Lafuente (stronk7) added a comment - Hi Petr, great job all the OpenDocument stuff! Related with 1) I've sent to HEAD the modified lib/excelib.class.php file that, based on the $CFG->excelisofiles (true/false) value, will generate iso-8859-1 files that are stronger than the default utf-16 ones. I know that this setting is only useful for iso-8859-1 people but, well, they are a lot TODO: a) Add $CFG->excelisofiles somewhere in the admin interface. b) Backport the changes to 17_STABLE (in real world I've tested my changes against two moodle servers running 1.7 and results were 100% successful with some sheets not being properly generated. It would be great if you can do it. I'll be out the next 10 days (27th to 4th).... Ciao
            Hide
            skodak Petr Skoda added a comment -

            OK, no problem - I will implement it

            Show
            skodak Petr Skoda added a comment - OK, no problem - I will implement it
            Hide
            stronk7 Eloy Lafuente (stronk7) added a comment -

            Oki, I'm back-porting the library to 1.6 and 1.7 and adding the config interface to 1.8 and HEAD.

            Show
            stronk7 Eloy Lafuente (stronk7) added a comment - Oki, I'm back-porting the library to 1.6 and 1.7 and adding the config interface to 1.8 and HEAD.
            Hide
            stronk7 Eloy Lafuente (stronk7) added a comment -

            Updated excel library is now in 16_STABLE, 17_STABLE, 18_STABLE and HEAD. Now it encodes to windows-1252 because if adds support for some more characters not present in iso-8859-1.

            Also, although working, for correctness, the $CFG->excelisofiles setting although working have been deprecated in favour of $CFG->latinexcelexport.

            Next step: Admin interface for this setting 18_STABLE and HEAD.

            Show
            stronk7 Eloy Lafuente (stronk7) added a comment - Updated excel library is now in 16_STABLE, 17_STABLE, 18_STABLE and HEAD. Now it encodes to windows-1252 because if adds support for some more characters not present in iso-8859-1. Also, although working, for correctness, the $CFG->excelisofiles setting although working have been deprecated in favour of $CFG->latinexcelexport. Next step: Admin interface for this setting 18_STABLE and HEAD.
            Hide
            stronk7 Eloy Lafuente (stronk7) added a comment -

            Subtask MDL-8585 created for HQ in order to decide where to implement the setting....

            Show
            stronk7 Eloy Lafuente (stronk7) added a comment - Subtask MDL-8585 created for HQ in order to decide where to implement the setting....
            Hide
            dougiamas Martin Dougiamas added a comment -

            Looks great! Marking as fixed.

            Show
            dougiamas Martin Dougiamas added a comment - Looks great! Marking as fixed.

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:
                  Fix Release Date:
                  30/Mar/07