Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-10637

BOM markers in some Moodle files

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Trivial
    • Resolution: Fixed
    • Affects Version/s: 1.9
    • Fix Version/s: 1.9
    • Component/s: Unicode
    • Labels:
      None
    • Environment:
      Linux
    • Affected Branches:
      MOODLE_19_STABLE
    • Fixed Branches:
      MOODLE_19_STABLE

      Description

      Some moodle files appear to have BOM markers in them. Under linux, they can be seen at the top left of some pages. They look like this:

      

      I've seen them in the 1.9 checkout I've made for testing, in one of the install steps, and on the Network Settings admin page at least. I suspect if there are BOM markers on those files there may be more elsewhere.

        Gliffy Diagrams

          Attachments

            Issue Links

              Activity

              nigel Nigel McNie created issue -
              Hide
              stronk7 Eloy Lafuente (stronk7) added a comment - - edited

              Executing this under Moodle dir:

              grep -rl $'\xEF\xBB\xBF' *

              should return all the files with the BOM.

              Once ignored all the binary files, each text file (php, xml...) should be processed with some thing like:

              vim FILE

              :set nobomb

              (save and exit)

              Don't forget to cvs diff FILE to check that only the 1st line (1st character) has changed.

              Finally, I've attached one shell script (bom_out.sh - only tested under MacOS X) that will search and destroy all the BOMs under directory where it's executed.

              Please, before committing, execute one "cvs diff" to check that only BOMs have been deleted!!

              Hope this helps...ciao

              Show
              stronk7 Eloy Lafuente (stronk7) added a comment - - edited Executing this under Moodle dir: grep -rl $'\xEF\xBB\xBF' * should return all the files with the BOM. Once ignored all the binary files, each text file (php, xml...) should be processed with some thing like: vim FILE :set nobomb (save and exit) Don't forget to cvs diff FILE to check that only the 1st line (1st character) has changed. Finally, I've attached one shell script (bom_out.sh - only tested under MacOS X) that will search and destroy all the BOMs under directory where it's executed. Please, before committing, execute one "cvs diff" to check that only BOMs have been deleted!! Hope this helps...ciao
              Hide
              lazyfish Yu Zhang added a comment -

              I think Nic is already working on this. Assigning to Nic.

              Show
              lazyfish Yu Zhang added a comment - I think Nic is already working on this. Assigning to Nic.
              lazyfish Yu Zhang made changes -
              Field Original Value New Value
              Assignee Yu Zhang [ lazyfish ] Nicolas Connault [ nicolasconnault ]
              Hide
              nicolasconnault Nicolas Connault added a comment -

              I ran the grep search on windows, no results. Martin ran it on linux, no results either. Have you actually tried this?

              Show
              nicolasconnault Nicolas Connault added a comment - I ran the grep search on windows, no results. Martin ran it on linux, no results either. Have you actually tried this?
              Hide
              stronk7 Eloy Lafuente (stronk7) added a comment -

              Perhaps it's Mac OS X only featute. Here it's the list:

              grep -rl $'\xEF\xBB\xBF' *

              lang/en_utf8/admin.php
              lang/en_utf8/help/courseenrollable2.html
              lang/en_utf8/help/courseformats.html
              lib/adodb/lang/adodb-bgutf8.inc.php
              lib/editor/tinymce/jscripts/tiny_mce/langs/ru_UTF-8.js
              lib/editor/tinymce/jscripts/tiny_mce/langs/zh_tw_utf8.js
              lib/tcpdf/fonts/FreeMonoBoldOblique.z
              theme/cornflower/images/classroomrevolution.jpg
              theme/cornflower/images/hdbg.jpg
              theme/cornflower/images/hdbgend.jpg
              theme/cornflower/images/logo.psd

              (obviously, binaries can be skipped)

              Show
              stronk7 Eloy Lafuente (stronk7) added a comment - Perhaps it's Mac OS X only featute. Here it's the list: grep -rl $'\xEF\xBB\xBF' * lang/en_utf8/admin.php lang/en_utf8/help/courseenrollable2.html lang/en_utf8/help/courseformats.html lib/adodb/lang/adodb-bgutf8.inc.php lib/editor/tinymce/jscripts/tiny_mce/langs/ru_UTF-8.js lib/editor/tinymce/jscripts/tiny_mce/langs/zh_tw_utf8.js lib/tcpdf/fonts/FreeMonoBoldOblique.z theme/cornflower/images/classroomrevolution.jpg theme/cornflower/images/hdbg.jpg theme/cornflower/images/hdbgend.jpg theme/cornflower/images/logo.psd (obviously, binaries can be skipped)
              Hide
              nicolasconnault Nicolas Connault added a comment -

              Thanks for the tips, Eloy!

              Show
              nicolasconnault Nicolas Connault added a comment - Thanks for the tips, Eloy!
              nicolasconnault Nicolas Connault made changes -
              Status Open [ 1 ] Resolved [ 5 ]
              Fix Version/s 1.9 [ 10190 ]
              Resolution Fixed [ 1 ]
              Hide
              stronk7 Eloy Lafuente (stronk7) added a comment -

              I've detected that some of the files were lang files and the utf8=>iso=>utf8 conversion trick isn't usable at all there (in fact it break files).

              I've fixed them now in CVS and edited the initial message with a simpler and safer way to delete those BOM characters with "vi".

              Ciao

              Show
              stronk7 Eloy Lafuente (stronk7) added a comment - I've detected that some of the files were lang files and the utf8=>iso=>utf8 conversion trick isn't usable at all there (in fact it break files). I've fixed them now in CVS and edited the initial message with a simpler and safer way to delete those BOM characters with "vi". Ciao
              poltawski Dan Poltawski made changes -
              Link This issue has been marked as being related by MDL-10753 [ MDL-10753 ]
              stronk7 Eloy Lafuente (stronk7) made changes -
              Attachment bom_out.sh [ 11656 ]
              Hide
              stronk7 Eloy Lafuente (stronk7) added a comment -

              Attached one shell script (bom_out.sh) that will, recursively, kill all BOMs. Only tested in Mac OS X.

              Please, review changes before commit them.

              Ciao

              Show
              stronk7 Eloy Lafuente (stronk7) added a comment - Attached one shell script (bom_out.sh) that will, recursively, kill all BOMs. Only tested in Mac OS X. Please, review changes before commit them. Ciao
              stronk7 Eloy Lafuente (stronk7) made changes -
              Attachment bom_out.sh [ 11656 ]
              stronk7 Eloy Lafuente (stronk7) made changes -
              Attachment bom_out.sh [ 11660 ]
              dougiamas Martin Dougiamas made changes -
              Status Resolved [ 5 ] Closed [ 6 ]
              QA Assignee nobody
              dougiamas Martin Dougiamas made changes -
              Workflow jira [ 21731 ] MDL Workflow [ 56653 ]
              dougiamas Martin Dougiamas made changes -
              Workflow MDL Workflow [ 56653 ] MDL Full Workflow [ 85781 ]
              howardsmiller Howard Miller made changes -
              Link This issue has a non-specific relationship to CONTRIB-3359 [ CONTRIB-3359 ]

                People

                • Votes:
                  0 Vote for this issue
                  Watchers:
                  1 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved:
                    Fix Release Date:
                    3/Mar/08