Is there a way to remove the BOM from a UTF-8 encoded file?

Question

Is there a way to remove the BOM from a UTF-8 encoded file?

I know that all of my JSON files are encoded in UTF-8, but the data entry person who edited the JSON files saved it as UTF-8 with the BOM.

When I run my Ruby scripts to parse the JSON, it is failing with an error. I don't want to manually open 58+ JSON files and convert to UTF-8 without the BOM.

Answer 1

With ruby >= 1.9.2 you can use the mode r:bom|utf-8

This should work (I haven't test it in combination with json):

json = nil #define the variable outside the block to keep the data File.open('file.txt', "r:bom|utf-8"){|file|   json = JSON.parse(file.read) } 

It doesn't matter, if the BOM is available in the file or not.

---

Andrew remarked, that File#rewind can't be used with BOM.

If you need a rewind-function you must remember the position and replace rewind with pos=:

#Prepare test file File.open('file.txt', "w:utf-8"){|f|   f << "\xEF\xBB\xBF" #add BOM   f << 'some content' }  #Read file and skip BOM if available File.open('file.txt', "r:bom|utf-8"){|f|   pos =f.pos   p content = f.read  #read and write file content   f.pos = pos   #f.rewind  goes to pos 0   p content = f.read  #(re)read and write file content }