Encoding Undefined Conversion Error - Ruby on Rails , Error on Write

Question

So I am trying to download a zip file from a website and put that data into a table. The download has been working before, but suddenly, it's not anymore. I get a "`write': "\xB6" from ASCII-8BIT to UTF-8 (Encoding::UndefinedConversionError)" problem. Very strange, nothing has changed since i last ran the script. Here is the part of the code that is having the problems. I am using a gem which allows me to parse the DBF file within the zip file.

Here's the code:

`write': "\xB6" from ASCII-8BIT to UTF-8 (Encoding::UndefinedConversionError)

Thanks for the help

UPDATE: I'm running within rails, obviously. I ran the script outside the rails environment, and it worked fine. So Rails is interfering somehow with the write process.

UPDATE 2: SOLVED - I changed "w" to "wb" apparently this is caused by rails 3+. Didn't occur to me to try this before. Hopefully this helps other people.

Answer 1

You probably also could have done it by passing

:output_encoding => "binary"

to File.open, which I think would do the same thing as mode 'wb', perhaps more clearly. But nice job figuring out 'wb' :) .

Yeah, in ruby 1.9, which is char encoding aware, you often need to tell different things about the encoding you want, including the "null encoding" 'binary'.

It's possible Rails is setting the Encoding.default_external to UTF-8, and File is using Encoding.default_external, that could possibly be why rails vs not-rails makes a difference.

Encoding issues in ruby 1.9 can definitely get very confusing very fast, with lots of normally hidden state that can affect exactly what happens.