Ruby Gem randomly returns Encoding Error

Question

So I forked this gem on GitHub, thinking that I may be able to fix and update some of the issues with it for use in a Rails project. I basically get this output:

irb(main):020:0> query = Query::simpleQuery('xx.xxx.xxx.xx', 25565)
=> [false, #<Encoding::CompatibilityError: incompatible character encodings: UTF-8 and ASCII-8BIT>]

irb(main):021:0> query = Query::simpleQuery('xx.xxx.xxx.xx', 25565)
=> {:motd=>"Craftnet", :gametype=>"SMP", :map=>"world", :numplayers=>"0", :maxplayers=>"48"}

The first response is the example of the Encoding error, and the second is the wanted output (IP's taken out). Basically this is querying a Minecraft server for information on it.

I tried using

Encoding.default_external = Encoding::UTF_8
Encoding.default_internal = Encoding::UTF_8

But that just gave the same response, randomly spitting encoding errors and not.

Here is the relevant GitHub repo with all the code: RubyMinecraft

Any help would be greatly appreciated.

Answer 1

In the Query class there is this line:

@key = Array(key).pack('N')

This creates a String with an associated encoding of ASCII-8BIT (i.e. it’s a binary string).

Later @key gets used in this line:

query = @sock.send("\xFE\xFD\x00\x01\x02\x03\x04" + @key, 0)

In Ruby 2.0 the default encoding of String literals is UTF-8, so this is combining a UTF-8 string with a binary one.

When Ruby tries to do this it first checks to see if the binary string only contains 7-bit values (i.e. all bytes are less than or equal to 127, with the top byte being 0), and if it does it considers it compatible with UTF-8 and so combines them without further issue. If it doesn’t, (i.e. if it contains bytes greater than 127) then the two strings are not compatible and an Encoding::CompatibilityError is raised.

Whether an error is raised depends on the contents of @key, which is initialized from a response from the server. Sometimes this value happens to contain only 7-bit values, so no error is raised, at other times there is a byte with the high bit set, so it generates an error. This is why the errors appear to be “random”.

To fix it you can specify that the string literal in the line where the two strings are combined should be treated as binary. The simplest way would be to use force_encoding like this:

query = @sock.send("\xFE\xFD\x00\x01\x02\x03\x04".force_encoding(Encoding::ASCII_8BIT) + @key, 0)