Moodle
  1. Moodle
  2. MDL-33753

moodle unzipping is not utf-8 compatible

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.2, 2.3
    • Fix Version/s: 2.4
    • Component/s: General
    • Labels:
    • Testing Instructions:
      Hide

      1/ run phpunit tests (it includes tests of the most common zip utils)
      2/ try to unzip random zip files with utf-8 chars - please note you may need to change moodle language to match Windows default non-unicode locale

      Show
      1/ run phpunit tests (it includes tests of the most common zip utils) 2/ try to unzip random zip files with utf-8 chars - please note you may need to change moodle language to match Windows default non-unicode locale
    • Affected Branches:
      MOODLE_22_STABLE, MOODLE_23_STABLE
    • Fixed Branches:
      MOODLE_24_STABLE
    • Pull from Repository:
    • Pull Master Branch:
      w28_MDL-33753_m24_utfunzip
    • Rank:
      41787

      Description

      At present we can only unzip archives created with unicode filenames - but the standard is to use unicode flag or extra attribute for utf-8 names, it is also expected to fall back to dos charset.

        Issue Links

          Activity

          Hide
          Dan Poltawski added a comment -

          Taking integration held issues out of integration (whilst we are keeping master and 23_STABLE in sync).

          Show
          Dan Poltawski added a comment - Taking integration held issues out of integration (whilst we are keeping master and 23_STABLE in sync).
          Hide
          Sam Hemelryk added a comment -

          Again changes look good and appear to work perfectly thanks Petr. Has been integrated now.

          Show
          Sam Hemelryk added a comment - Again changes look good and appear to work perfectly thanks Petr. Has been integrated now.
          Hide
          Petr Škoda added a comment -

          arrgggh, I have fixed a sloppy zipping regression caused by refactoring in this issue, please merge.

          Show
          Petr Škoda added a comment - arrgggh, I have fixed a sloppy zipping regression caused by refactoring in this issue, please merge.
          Hide
          Dan Poltawski added a comment -

          Integrated, thanks

          Show
          Dan Poltawski added a comment - Integrated, thanks
          Hide
          Rossiani Wijaya added a comment -

          This looks great.
          Thank for the quick fix.

          Test passed.

          Show
          Rossiani Wijaya added a comment - This looks great. Thank for the quick fix. Test passed.
          Hide
          Dan Poltawski added a comment -

          Congratulations!

          You've made it into the weekly release!

          Thanks for your contribution - here are some random drummers to keep you inspired for the next week!
          http://www.youtube.com/watch?v=_QhpHUmVCmY

          Show
          Dan Poltawski added a comment - Congratulations! You've made it into the weekly release! Thanks for your contribution - here are some random drummers to keep you inspired for the next week! http://www.youtube.com/watch?v=_QhpHUmVCmY
          Show
          Vadim Dvorovenko added a comment - - edited 2.2 & 2.3 backport https://github.com/vadimonus/moodle/moodle.git Branches: MDL-33710 -33753-22 & MDL-33710 -33753-23 Diff: https://github.com/vadimonus/moodle/compare/4db36990ee032eef3b46d06179e85595fade3fbd...MDL-33710-33753-22 https://github.com/vadimonus/moodle/compare/32efb09e62b4a68299c6c9eaafe001cce517e5d0...MDL-33710-33753-23
          Hide
          miki Alliel added a comment -

          This is the zip file that I was trying to upload to moodle and to extract but the Hebrew filnames were disappeared from the filenames

          Show
          miki Alliel added a comment - This is the zip file that I was trying to upload to moodle and to extract but the Hebrew filnames were disappeared from the filenames
          Hide
          miki Alliel added a comment -

          Gebbrish filename instead of Hebrew filename after extracting zip file with assignments

          Show
          miki Alliel added a comment - Gebbrish filename instead of Hebrew filename after extracting zip file with assignments
          Hide
          miki Alliel added a comment -

          Hello
          I checked this on Moodle 2.3.2+ (Build: 20120927)
          I uploaded zip file (attached here) and extract it in moodle site the file contains 2 files which the name of the files are in Hebrew (utf8 lang) but after extracting the zip I saw that the names disappeared (infact one file includes hebrew name and a number (1) so only the number (1) stayed as the filename).

          Another problem as shown in the images attached , after clicking on "Download all assignments as a zip" the files with Hebrew names in the zip where Gibbrish!

          Hope there is solution
          Thanks

          Show
          miki Alliel added a comment - Hello I checked this on Moodle 2.3.2+ (Build: 20120927) I uploaded zip file (attached here) and extract it in moodle site the file contains 2 files which the name of the files are in Hebrew (utf8 lang) but after extracting the zip I saw that the names disappeared (infact one file includes hebrew name and a number (1) so only the number (1) stayed as the filename). Another problem as shown in the images attached , after clicking on "Download all assignments as a zip" the files with Hebrew names in the zip where Gibbrish! Hope there is solution Thanks
          Hide
          Nadav Kavalerchik added a comment -

          I can confirm Miki's comments above when tested on an updated master branch (from 27-9-2012)

          Show
          Nadav Kavalerchik added a comment - I can confirm Miki's comments above when tested on an updated master branch (from 27-9-2012)
          Hide
          Petr Škoda added a comment -

          Hi, the attached file is not unicode compatible, sorry, you should use some different packer instead - such as 7zip. The built-in zipping in Windows OS is horrible.

          I am going to add Hebrew DOs charset fallback in MDL-35795, thanks for the sample zip file.

          Show
          Petr Škoda added a comment - Hi, the attached file is not unicode compatible, sorry, you should use some different packer instead - such as 7zip. The built-in zipping in Windows OS is horrible. I am going to add Hebrew DOs charset fallback in MDL-35795 , thanks for the sample zip file.
          Hide
          Petr Škoda added a comment -

          Could somebody please test my patch in MDL-35795? I have uploaded the attached file there too.

          Show
          Petr Škoda added a comment - Could somebody please test my patch in MDL-35795 ? I have uploaded the attached file there too.
          Hide
          Nadav Kavalerchik added a comment - - edited

          Petr,

          "the attached file is not unicode compatible"
          I know. that's way I attached it. so you can see what we have to deal with
          "The built-in zipping in Windows OS is horrible."
          Tell me about it

          Here, in the middle east, there is no way we can tell the teachers to use anything. (you can watch the News and see why)
          We are lucky if they use Chrome or Firefox and not IE.
          Most computer labs at School and Teacher Training Colleges are "managed" (locked down) and you can not install 7Zip.
          (I have tried. even wrote feature comparison document. but, nothing helps)

          We need to try to fix it on the server side. If we can.
          MDL-33068 Seems to do the trick. In addition, we should create a system setting for a default encoding. So the Teachers do not have to understand it and decide what to choose. And have the Administrator set it up for them. (In our, Hebrew, case it is WINDOWS-1255 or ISO-8859-8)

          Also, I have tested your beautiful solution on MDL-35795, which works great too!

          One of these solution should be implemented everywhere there is an option to download or upload a zip file ( File, Folder, Assignments... )

          Show
          Nadav Kavalerchik added a comment - - edited Petr, "the attached file is not unicode compatible" I know. that's way I attached it. so you can see what we have to deal with "The built-in zipping in Windows OS is horrible." Tell me about it Here, in the middle east, there is no way we can tell the teachers to use anything. (you can watch the News and see why) We are lucky if they use Chrome or Firefox and not IE. Most computer labs at School and Teacher Training Colleges are "managed" (locked down) and you can not install 7Zip. (I have tried. even wrote feature comparison document. but, nothing helps) We need to try to fix it on the server side. If we can. MDL-33068 Seems to do the trick. In addition, we should create a system setting for a default encoding. So the Teachers do not have to understand it and decide what to choose. And have the Administrator set it up for them. (In our, Hebrew, case it is WINDOWS-1255 or ISO-8859-8) Also, I have tested your beautiful solution on MDL-35795 , which works great too! One of these solution should be implemented everywhere there is an option to download or upload a zip file ( File, Folder, Assignments... )
          Hide
          miki Alliel added a comment -

          downloadall in zip - Heb (utf8) file names are seen in Gibbrish after opening the zip file in windows

          Show
          miki Alliel added a comment - downloadall in zip - Heb (utf8) file names are seen in Gibbrish after opening the zip file in windows
          Hide
          miki Alliel added a comment -

          I'v just check this problem in this site
          http://qa.moodle.net

          which is moodle 2.4 as I understand.

          This problem still exists there!

          when downloading assignment in zip file (download all) the file names which were in Hebrew are in Gibberish when opening the zip file
          like the image I attach here
          thanks

          Show
          miki Alliel added a comment - I'v just check this problem in this site http://qa.moodle.net which is moodle 2.4 as I understand. This problem still exists there! when downloading assignment in zip file (download all) the file names which were in Hebrew are in Gibberish when opening the zip file like the image I attach here thanks
          Hide
          Petr Škoda added a comment -

          Windows does not support Unicode in zip archives at all, it is no wonder it shows everything as Gibberish. Please use some real zip extractor, not the broken built-in unzipping. It is able to extract only borked archives which were created in Windows with the same DOS encoding, nothing else.

          Show
          Petr Škoda added a comment - Windows does not support Unicode in zip archives at all, it is no wonder it shows everything as Gibberish. Please use some real zip extractor, not the broken built-in unzipping. It is able to extract only borked archives which were created in Windows with the same DOS encoding, nothing else.
          Hide
          miki Alliel added a comment -

          You are right , Im sorry,
          I just tried again to open this zip file with 7zip program ( on windows 7) and finally I saw the Hebrew file name letters.
          Thanks

          Show
          miki Alliel added a comment - You are right , Im sorry, I just tried again to open this zip file with 7zip program ( on windows 7) and finally I saw the Hebrew file name letters. Thanks

            People

            • Votes:
              3 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: