Json dumping bytes fails in Python 3

Question

I am sending binary data in a post request as part of a request. I have a dictionary that looks like this:

data = {"foo": "bar", "bar": b'foo'}

When I try to json.dumps this dictionary, I get the following exception:

TypeError: b'foo' is not JSON serializable

This worked fine in Python 2.7. What do I have to do to json encode this data?

Answer 1

In Python 3, they removed byte support in json. (Source: https://bugs.python.org/issue10976).

A possible workaround is:

import json

data = {"foo": "bar", "bar": b"foo"}

# decode the `byte` into a unicode `str`
data["bar"] = data["bar"].decode("utf8")

# `data` now contains
#
#   {'bar': 'foo', 'foo': 'bar'}
#
# `json_encoded_data` contains
#
#   '{"bar": "foo", "foo": "bar"}'
#
json_encoded_data = json.dumps(data)

# `json_decoded_data` contains
#
#   {'bar': 'foo', 'foo': 'bar'}
#
json_decoded_data = json.loads(data)

# `data` now contains
#
#   {'bar': b'foo', 'foo': 'bar'}
#
data["bar"] = data["bar"].encode("utf8")

If you don't have a constraint of using json, you might consider using bson (Binary JSON):

import bson

data = {"foo": "bar", "bar": b"foo"}

# `bson_encoded_data` contains 
#
#   b'\x1f\x00\x00\x00\x05bar\x00\x03\x00\x00\x00\x00foo\x02foo\x00\x04\x00\x00\x00bar\x00\x00'
#
bson_encoded_data = bson.BSON.encode(data)

# `bson_decoded_data` contains
#
#   {'bar': b'foo', 'foo': 'bar'}
#
bson_decoded_data = bson.BSON.decode(data)

Answer 2

With Json module you cannot dump bytes. A suitable alternative is to use Simple Json Module.

To Install Simple json:

pip3 install simplejson

Code:

import simplejson as json
data = {"foo": "bar", "bar": b"foo"}
json.dumps(data)

Hopefully you won't get the error now!