Decoding msgpack_numpy with utf-8 strings

Question

I use python 3.6 with msgpack==0.5.1 and msgpack_numpy==0.4.2.

When trying to encode and decode a dict, the string needs to be handled using utf-8 to restore the dict's keys as strings (instead of binaries).

For example:

import msgpack

d = {'key': None}
binary = msgpack.packb(d)

ret = msgpack.unpackb(binary)
ret.keys()
>>> dict_keys([b'key'])

ret = msgpack.unpackb(binary, encoding='utf-8')
ret.keys()
>>> dict_keys(['key'])

However, when using msgpack_numpy, passing encoding='utf-8' brakes the numpy decoding:

import numpy as np
import msgpack_numpy as m
m.patch()

d['key'] = np.arange(5)
binary = msgpack.packb(d)

ret = msgpack.unpackb(binary)
ret.keys()
>>> dict_keys([b'key'])
ret[b'key']
>>> array([0, 1, 2, 3, 4])

ret = msgpack.unpackb(binary, encoding='utf-8')
ret.keys()
>>> dict_keys(['key'])
ret['key']
>>> {'data': '\x00\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x02\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00', 'kind': '', 'nd': True, 'shape': [5], 'type': '<i8'}

Is it possible to encode/decode numpy arrays using msgpack without replacing the dict's keys to binary?

Answer 1

I fiddled with different packing options and discovered that using use_bin_type=True when packing the object solves the problem.

import msgpack
import numpy as np
import msgpack_numpy as m
m.patch()

d = {'key': np.arange(5)}
binary = msgpack.packb(d, use_bin_type=True)

ret = msgpack.unpackb(binary, encoding='utf-8')
ret.keys()
>>> dict_keys(['key'])
ret['key']
>>> array([0, 1, 2, 3, 4])