python 3 will not find key in dict from msgpack

Question

why does this happen in python3?

1) I get msgpack data from redis

2) I unpack I get the below

3) The returned type is a dict:

meta = msgpack.unpackb(data[1])
print(type(meta))
<class 'dict'>

 meta = {b'api_key': b'apikey1',
        b'sensor_id': b'sid1',
        b'version': b'1.0'}

If I run the below: sensor_meta['sensor_id']

{b'api_key': b'apikey1',
 b'sensor_id': b'sid1',
 b'version': b'1.0'}
Traceback (most recent call last):
  File "/Users//worker.py", line 247, in <module>
    print(meta['sensor_id'])
KeyError: 'sensor_id'

but if I use sensor_meta[b'sensor_id'] then it works.

What is the "b" and how can i get rid of that? How do I convert the whole object so there are no b's ?

so if I do the below:

   print(type(meta['sensor_id']))
   <class 'bytes'>

why bytes and how did it get there? I do not to append a b for every time I want to use keys in a hash.

Thanks

Answer 1

As mentioned in the notes here:

string and binary type In old days, msgpack doesn’t distinguish string and binary types like Python 1. The type for represent string and binary types is named raw.

msgpack can distinguish string and binary type for now. But it is not like Python 2. Python 2 added unicode string. But msgpack renamed raw to str and added bin type. It is because keep compatibility with data created by old libs. raw was used for text more than binary.

Currently, while msgpack-python supports new bin type, default setting doesn’t use it and decodes raw as bytes instead of unicode (str in Python 3).

You can change this by using use_bin_type=True option in Packer and encoding=”utf-8” option in Unpacker.

>>> import msgpack
>>> packed = msgpack.packb([b'spam', u'egg'], use_bin_type=True)
>>> msgpack.unpackb(packed, encoding='utf-8') ['spam', u'egg']

You can define an encoding while unpacking to convert your bytes to strings.

msgpack.unpackb(data[1], encoding='utf-8')