ruby `encode': "\xC3" from ASCII-8BIT to UTF-8 (Encoding::UndefinedConversionError)

Question

It seems you should use another encoding for the object. You should set the proper codepage to the variable @tree, for instance, using iso-8859-1 instead of ascii-8bit by using @tree.force_encoding('ISO-8859-1'). Because ASCII-8BIT is used just for binary files.

To find the current external encoding for ruby, issue:

Encoding.default_external

If sudo solves the problem, the problem was in default codepage (encoding), so to resolve it you have to set the proper default codepage (encoding), by either:

1. In ruby to change encoding to utf-8 or another proper one, do as follows:

   Encoding.default_external = Encoding::UTF_8
   

2. In bash, grep current valid set up:

   $ sudo env|grep UTF-8
   LC_ALL=ru_RU.UTF-8
   LANG=ru_RU.UTF-8
   

   Then set them in .bashrc properly, in a similar way, but not exactly with ru_RU language, such as the following:

   export LC_ALL=ru_RU.UTF-8
   export LANG=ru_RU.UTF-8
   

File.open(yml_file, 'w') should be change to File.open(yml_file, 'wb')

I just suffered through a number of hours trying to fix a similar problem. I'd checked my locales, database encoding, everything I could think of and was still getting ASCII-8BIT encoded data from the database.

Well, it turns out that if you store text in a binary field, it will automatically be returned as ASCII-8BIT encoded text, which makes sense, however this can (obviously) cause problems in your application.

It can be fixed by changing the column encoding back to :text in your migrations.

I had the same problems when saving to the database. I'll offer one thing that I use (perhaps, this will help someone).

if you know that sometimes your text has strange characters, then before saving you can encode your text in some other format, and then decode the text again after it is returned from the database.

example:

string = "Œuf"

before save we encode string

text_to_save = CGI.escape(string)

(character "Œ" encoded in "%C5%92" and other characters remained the same)

=> "%C5%92uf"

load from database and decode

CGI.unescape("%C5%92uf")

=> "Œuf"