Storing a MessagePacked hash in Redis

Question

I'm having a problem storing a MessagePacked hash in Redis. I've pasted a test case below. When pulling out the packed data from Redis and unpacking it, the hash is slightly corrupted. This appears to happen when the hash values are beyond a certain length, although I can't say that for sure.

I'm using Redis 2.4.17 (default config), Ruby 1.9.3p194, MessagePack 0.4.7, and the Redis gem 3.0.2. The same problem happens using node, so I'm assuming the problem is within MessagePack or Redis. Any ideas?

require 'redis'
require 'msgpack'

class Test

  def self.run(url)
    redis = Redis.new
    data = {'number' => 13498935756, 'hash' => {'url' => url}}
    redis.set('my_key', MessagePack.pack(data))

    result = MessagePack.unpack(redis.get('my_key'))
    puts result
    puts result['hash']['url'] == data['hash']['url']
  end

end

Test.run('http://fake.example.com')  # works
=>  {"number"=>13498935756, "hash"=>{"url"=>"http://fake.example.com"}}
=>  true

Test.run('http://fakeurl.example.com')  # does not work
=>  {"number"=>13498935756, "hash"=>{"url"=>"ttp://fakeurl.example.com"}}
=>  false

Answer 1

MessagePack deals in raw bytes, which are marked as 'ASCII-8BIT' encoding. However your packed data is coming back from Redis marked as being in UTF-8 encoding. In order for MessagePack to successfully unpack, you need to force it back to being interpreted as raw bytes.

Therefore, change this line...

result = MessagePack.unpack(redis.get('my_key'))

to something like this...

redis_val = redis.get('my_key').force_encoding('ASCII-8BIT')
result = MessagePack.unpack(redis_val)