Issue Details (XML | Word | Printable)

Key: MDL-10637
Type: Bug Bug
Status: Resolved Resolved
Resolution: Fixed
Priority: Trivial Trivial
Assignee: Nicolas Connault
Reporter: Nigel McNie
Votes: 0
Watchers: 2
Operations

Add/Edit UI Mockup to this issue
If you were logged in you would be able to see more operations.
Moodle

BOM markers in some Moodle files

Created: 28/Jul/07 08:05 PM   Updated: 10/Aug/07 06:24 AM
Return to search
Component/s: Unicode
Affects Version/s: 1.9
Fix Version/s: 1.9

File Attachments: 1. File bom_out.sh (0.2 kB)

Environment: Linux
Issue Links:
Relates
 

Participants: Eloy Lafuente (stronk7), Nicolas Connault, Nigel McNie and Yu Zhang
Security Level: None
Resolved date: 30/Jul/07
Affected Branches: MOODLE_19_STABLE
Fixed Branches: MOODLE_19_STABLE

Sub-Tasks  All   Open   
 Sub-Task Progress: 
No sub-tasks match this view.

 Description  « Hide
Some moodle files appear to have BOM markers in them. Under linux, they can be seen at the top left of some pages. They look like this:



I've seen them in the 1.9 checkout I've made for testing, in one of the install steps, and on the Network Settings admin page at least. I suspect if there are BOM markers on those files there may be more elsewhere.

 All   Comments   Change History   Version Control      Sort Order: Ascending order - Click to sort in descending order
Eloy Lafuente (stronk7) added a comment - 28/Jul/07 11:57 PM - edited
Executing this under Moodle dir:

grep -rl $'\xEF\xBB\xBF' *

should return all the files with the BOM.

Once ignored all the binary files, each text file (php, xml...) should be processed with some thing like:

vim FILE

:set nobomb

(save and exit)

Don't forget to cvs diff FILE to check that only the 1st line (1st character) has changed.

Finally, I've attached one shell script (bom_out.sh - only tested under MacOS X) that will search and destroy all the BOMs under directory where it's executed.

Please, before committing, execute one "cvs diff" to check that only BOMs have been deleted!!

Hope this helps...ciao


Yu Zhang added a comment - 30/Jul/07 09:54 AM
I think Nic is already working on this. Assigning to Nic.

Yu Zhang made changes - 30/Jul/07 09:54 AM
Field Original Value New Value
Assignee Yu Zhang [ lazyfish ] Nicolas Connault [ nicolasconnault ]
Nicolas Connault added a comment - 30/Jul/07 01:56 PM
I ran the grep search on windows, no results. Martin ran it on linux, no results either. Have you actually tried this?

Eloy Lafuente (stronk7) added a comment - 30/Jul/07 03:21 PM
Perhaps it's Mac OS X only featute. Here it's the list:

grep -rl $'\xEF\xBB\xBF' *

lang/en_utf8/admin.php
lang/en_utf8/help/courseenrollable2.html
lang/en_utf8/help/courseformats.html
lib/adodb/lang/adodb-bgutf8.inc.php
lib/editor/tinymce/jscripts/tiny_mce/langs/ru_UTF-8.js
lib/editor/tinymce/jscripts/tiny_mce/langs/zh_tw_utf8.js
lib/tcpdf/fonts/FreeMonoBoldOblique.z
theme/cornflower/images/classroomrevolution.jpg
theme/cornflower/images/hdbg.jpg
theme/cornflower/images/hdbgend.jpg
theme/cornflower/images/logo.psd

(obviously, binaries can be skipped)


Nicolas Connault committed 5 files to 'Moodle CVS' - 30/Jul/07 07:51 PM
MDL-10637 Applied Eloy's suggested fix to the BOMs in the files he detected on his oh-so-aware Mac :-)
MODIFY lang/en_utf8/help/courseformats.html   Rev. 1.5    (+1 -1 lines)
MODIFY lib/editor/tinymce/jscripts/tiny_mce/langs/Attic/ru_UTF-8.js   Rev. 1.2    (+35 -35 lines)
MODIFY lib/adodb/lang/adodb-bgutf8.inc.php   Rev. 1.2    (+27 -27 lines)
MODIFY lib/editor/tinymce/jscripts/tiny_mce/langs/Attic/zh_tw_utf8.js   Rev. 1.2    (+0 -42 lines)
MODIFY lang/en_utf8/help/courseenrollable2.html   Rev. 1.2    (+1 -1 lines)
Nicolas Connault added a comment - 30/Jul/07 07:54 PM
Thanks for the tips, Eloy!

Nicolas Connault made changes - 30/Jul/07 07:54 PM
Fix Version/s 1.9 [ 10190 ]
Resolution Fixed [ 1 ]
Status Open [ 1 ] Resolved [ 5 ]
Eloy Lafuente (stronk7) added a comment - 30/Jul/07 11:25 PM
I've detected that some of the files were lang files and the utf8=>iso=>utf8 conversion trick isn't usable at all there (in fact it break files).

I've fixed them now in CVS and edited the initial message with a simpler and safer way to delete those BOM characters with "vi".

Ciao


Dan Poltawski made changes - 09/Aug/07 03:42 AM
Link This issue has been marked as being related by MDL-10753 [ MDL-10753 ]
Eloy Lafuente (stronk7) made changes - 09/Aug/07 11:56 PM
Attachment bom_out.sh [ 11656 ]
Eloy Lafuente (stronk7) added a comment - 10/Aug/07 12:03 AM
Attached one shell script (bom_out.sh) that will, recursively, kill all BOMs. Only tested in Mac OS X.

Please, review changes before commit them.

Ciao


Eloy Lafuente (stronk7) made changes - 10/Aug/07 06:22 AM
Attachment bom_out.sh [ 11656 ]
Eloy Lafuente (stronk7) made changes - 10/Aug/07 06:24 AM
Attachment bom_out.sh [ 11660 ]