The regular way of JSON-serializing custom non-serializable objects is to subclass json.JSONEncoder
and then pass a custom encoder to json.dumps()
.
It usually looks like this:
class CustomEncoder(json.JSONEncoder): def default(self, obj): if isinstance(obj, Foo): return obj.to_json() return json.JSONEncoder.default(self, obj) print(json.dumps(obj, cls=CustomEncoder))
What I'm trying to do, is to make something serializable with the default encoder. I looked around but couldn't find anything. My thought is that there would be some field in which the encoder looks at to determine the json encoding. Something similar to __str__
. Perhaps a __json__
field. Is there something like this in python?
I want to make one class of a module I'm making to be JSON serializable to everyone that uses the package without them worrying about implementing their own [trivial] custom encoders.
Use toJSON() Method to make class JSON serializable So we don't need to write custom JSONEncoder. This new toJSON() serializer method will return the JSON representation of the Object. i.e., It will convert custom Python Object to JSON string.
The purpose of serializing it into JSON is so that the message will be a format that can be understood and from there, deserialize it into an object type that makes sense for the consumer.
JSON is a format that encodes objects in a string. Serialization means to convert an object into that string, and deserialization is its inverse operation (convert string -> object).
Serialization refers to the process of converting a data object (e.g., Python objects, Tensorflow models) into a format that allows us to store or transmit the data and then recreate the object when needed using the reverse process of deserialization.
As I said in a comment to your question, after looking at the json
module's source code, it does not appear to lend itself to doing what you want. However the goal could be achieved by what is known as monkey-patching (see question What is a monkey patch?). This could be done in your package's __init__.py
initialization script and would affect all subsequent json
module serialization since modules are generally only loaded once and the result is cached in sys.modules
.
The patch changes the default json encoder's default
method—the default default()
.
Here's an example implemented as a standalone module for simplicity's sake:
Module: make_json_serializable.py
""" Module that monkey-patches json module when it's imported so JSONEncoder.default() automatically checks for a special "to_json()" method and uses it to encode the object if found. """ from json import JSONEncoder def _default(self, obj): return getattr(obj.__class__, "to_json", _default.default)(obj) _default.default = JSONEncoder.default # Save unmodified default. JSONEncoder.default = _default # Replace it.
Using it is trivial since the patch is applied by simply importing the module.
Sample client script:
import json import make_json_serializable # apply monkey-patch class Foo(object): def __init__(self, name): self.name = name def to_json(self): # New special method. """ Convert to JSON format string representation. """ return '{"name": "%s"}' % self.name foo = Foo('sazpaz') print(json.dumps(foo)) # -> "{\"name\": \"sazpaz\"}"
To retain the object type information, the special method can also include it in the string returned:
return ('{"type": "%s", "name": "%s"}' % (self.__class__.__name__, self.name))
Which produces the following JSON that now includes the class name:
"{\"type\": \"Foo\", \"name\": \"sazpaz\"}"
Magick Lies Here
Even better than having the replacement default()
look for a specially named method, would be for it to be able to serialize most Python objects automatically, including user-defined class instances, without needing to add a special method. After researching a number of alternatives, the following — based on an answer by @Raymond Hettinger to another question — which uses the pickle
module, seemed closest to that ideal to me:
Module: make_json_serializable2.py
""" Module that imports the json module and monkey-patches it so JSONEncoder.default() automatically pickles any Python objects encountered that aren't standard JSON data types. """ from json import JSONEncoder import pickle def _default(self, obj): return {'_python_object': pickle.dumps(obj)} JSONEncoder.default = _default # Replace with the above.
Of course everything can't be pickled—extension types for example. However there are ways defined to handle them via the pickle protocol by writing special methods—similar to what you suggested and I described earlier—but doing that would likely be necessary for a far fewer number of cases.
Deserializing
Regardless, using the pickle protocol also means it would be fairly easy to reconstruct the original Python object by providing a custom object_hook
function argument on any json.loads()
calls that used any '_python_object'
key in the dictionary passed in, whenever it has one. Something like:
def as_python_object(dct): try: return pickle.loads(str(dct['_python_object'])) except KeyError: return dct pyobj = json.loads(json_str, object_hook=as_python_object)
If this has to be done in many places, it might be worthwhile to define a wrapper function that automatically supplied the extra keyword argument:
json_pkloads = functools.partial(json.loads, object_hook=as_python_object) pyobj = json_pkloads(json_str)
Naturally, this could be monkey-patched it into the json
module as well, making the function the default object_hook
(instead of None
).
I got the idea for using pickle
from an answer by Raymond Hettinger to another JSON serialization question, whom I consider exceptionally credible as well as an official source (as in Python core developer).
The code above does not work as shown in Python 3 because json.dumps()
returns a bytes
object which the JSONEncoder
can't handle. However the approach is still valid. A simple way to workaround the issue is to latin1
"decode" the value returned from pickle.dumps()
and then "encode" it from latin1
before passing it on to pickle.loads()
in the as_python_object()
function. This works because arbitrary binary strings are valid latin1
which can always be decoded to Unicode and then encoded back to the original string again (as pointed out in this answer by Sven Marnach).
(Although the following works fine in Python 2, the latin1
decoding and encoding it does is superfluous.)
from decimal import Decimal class PythonObjectEncoder(json.JSONEncoder): def default(self, obj): return {'_python_object': pickle.dumps(obj).decode('latin1')} def as_python_object(dct): try: return pickle.loads(dct['_python_object'].encode('latin1')) except KeyError: return dct class Foo(object): # Some user-defined class. def __init__(self, name): self.name = name def __eq__(self, other): if type(other) is type(self): # Instances of same class? return self.name == other.name return NotImplemented __hash__ = None data = [1,2,3, set(['knights', 'who', 'say', 'ni']), {'key':'value'}, Foo('Bar'), Decimal('3.141592653589793238462643383279502884197169')] j = json.dumps(data, cls=PythonObjectEncoder, indent=4) data2 = json.loads(j, object_hook=as_python_object) assert data == data2 # both should be same
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With