Moodle

BOM markers in some Moodle files

Details

  • Type: Bug Bug
  • Status: Closed Closed
  • Priority: Trivial Trivial
  • Resolution: Fixed
  • Affects Version/s: 1.9
  • Fix Version/s: 1.9
  • Component/s: Unicode
  • Labels:
    None
  • Environment:
    Linux
  • Affected Branches:
    MOODLE_19_STABLE
  • Fixed Branches:
    MOODLE_19_STABLE

Description

Some moodle files appear to have BOM markers in them. Under linux, they can be seen at the top left of some pages. They look like this:



I've seen them in the 1.9 checkout I've made for testing, in one of the install steps, and on the Network Settings admin page at least. I suspect if there are BOM markers on those files there may be more elsewhere.

Issue Links

This issue has a non-specific relationship to:
CONTRIB-3359 Turnitin adds BOM character into Gradebook export - this seems to cause corruption Critical Open
This issue has been marked as being related by:
MDL-10753 lang/admin.php Minor Closed

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Eloy Lafuente (stronk7) added a comment - - edited

Executing this under Moodle dir:

grep -rl $'\xEF\xBB\xBF' *

should return all the files with the BOM.

Once ignored all the binary files, each text file (php, xml...) should be processed with some thing like:

vim FILE

:set nobomb

(save and exit)

Don't forget to cvs diff FILE to check that only the 1st line (1st character) has changed.

Finally, I've attached one shell script (bom_out.sh - only tested under MacOS X) that will search and destroy all the BOMs under directory where it's executed.

Please, before committing, execute one "cvs diff" to check that only BOMs have been deleted!!

Hope this helps...ciao

Show
Eloy Lafuente (stronk7) added a comment - - edited Executing this under Moodle dir: grep -rl $'\xEF\xBB\xBF' * should return all the files with the BOM. Once ignored all the binary files, each text file (php, xml...) should be processed with some thing like: vim FILE :set nobomb (save and exit) Don't forget to cvs diff FILE to check that only the 1st line (1st character) has changed. Finally, I've attached one shell script (bom_out.sh - only tested under MacOS X) that will search and destroy all the BOMs under directory where it's executed. Please, before committing, execute one "cvs diff" to check that only BOMs have been deleted!! Hope this helps...ciao
Hide
Permalink
Yu Zhang added a comment -

I think Nic is already working on this. Assigning to Nic.

Show
Yu Zhang added a comment - I think Nic is already working on this. Assigning to Nic.
Hide
Permalink
Nicolas Connault added a comment -

I ran the grep search on windows, no results. Martin ran it on linux, no results either. Have you actually tried this?

Show
Nicolas Connault added a comment - I ran the grep search on windows, no results. Martin ran it on linux, no results either. Have you actually tried this?
Hide
Permalink
Eloy Lafuente (stronk7) added a comment -

Perhaps it's Mac OS X only featute. Here it's the list:

grep -rl $'\xEF\xBB\xBF' *

lang/en_utf8/admin.php
lang/en_utf8/help/courseenrollable2.html
lang/en_utf8/help/courseformats.html
lib/adodb/lang/adodb-bgutf8.inc.php
lib/editor/tinymce/jscripts/tiny_mce/langs/ru_UTF-8.js
lib/editor/tinymce/jscripts/tiny_mce/langs/zh_tw_utf8.js
lib/tcpdf/fonts/FreeMonoBoldOblique.z
theme/cornflower/images/classroomrevolution.jpg
theme/cornflower/images/hdbg.jpg
theme/cornflower/images/hdbgend.jpg
theme/cornflower/images/logo.psd

(obviously, binaries can be skipped)

Show
Eloy Lafuente (stronk7) added a comment - Perhaps it's Mac OS X only featute. Here it's the list: grep -rl $'\xEF\xBB\xBF' * lang/en_utf8/admin.php lang/en_utf8/help/courseenrollable2.html lang/en_utf8/help/courseformats.html lib/adodb/lang/adodb-bgutf8.inc.php lib/editor/tinymce/jscripts/tiny_mce/langs/ru_UTF-8.js lib/editor/tinymce/jscripts/tiny_mce/langs/zh_tw_utf8.js lib/tcpdf/fonts/FreeMonoBoldOblique.z theme/cornflower/images/classroomrevolution.jpg theme/cornflower/images/hdbg.jpg theme/cornflower/images/hdbgend.jpg theme/cornflower/images/logo.psd (obviously, binaries can be skipped)
Hide
Permalink
Nicolas Connault added a comment -

Thanks for the tips, Eloy!

Show
Nicolas Connault added a comment - Thanks for the tips, Eloy!
Hide
Permalink
Eloy Lafuente (stronk7) added a comment -

I've detected that some of the files were lang files and the utf8=>iso=>utf8 conversion trick isn't usable at all there (in fact it break files).

I've fixed them now in CVS and edited the initial message with a simpler and safer way to delete those BOM characters with "vi".

Ciao

Show
Eloy Lafuente (stronk7) added a comment - I've detected that some of the files were lang files and the utf8=>iso=>utf8 conversion trick isn't usable at all there (in fact it break files). I've fixed them now in CVS and edited the initial message with a simpler and safer way to delete those BOM characters with "vi". Ciao
Hide
Permalink
Eloy Lafuente (stronk7) added a comment -

Attached one shell script (bom_out.sh) that will, recursively, kill all BOMs. Only tested in Mac OS X.

Please, review changes before commit them.

Ciao

Show
Eloy Lafuente (stronk7) added a comment - Attached one shell script (bom_out.sh) that will, recursively, kill all BOMs. Only tested in Mac OS X. Please, review changes before commit them. Ciao

People

Vote (0)
Watch (1)

Dates

  • Created:
    Updated:
    Resolved:
Atlassian JIRA (v4.4.3#663-r165197) | Report a problem
Powered by a free Atlassian JIRA open source license for Moodle. Try JIRA - bug tracking software for your team.