python - using json with OrderedDict and Datetime

Question

I am using python to encode an OrderedDict with timestamp in it and I am having issues. The data that I am trying to encode looks like this:

OrderedDict([('a', datetime.datetime(2015, 6, 15, 15, 58, 54, 884000)), ('b', 'b'), ('c', 'c'), ('d', 'd')])

I expect this to be json encoded and decoded to get exactly the same data.

In order to encode timestamp directly, without changing to ISO or Unix time, I used bson's json_util interface as below. It works correctly.

json.dumps(str, default=json_util.default)
json.loads(jsonstr, object_hook=json_util.object_hook)

In order to get an OrderedDict I used object_pairs_hook, which also works:

json.loads(x, object_pairs_hook=OrderedDict)

However, when used together, the two things mess with each other and the result is not in the correct format(Since the bson interface is creating an extra dictionary for the timestamp).

json.loads(jsonstr, object_hook=json_util.object_hook, object_pairs_hook=OrderedDict)

This query ends up getting this:

OrderedDict([(u'a', OrderedDict([(u'$date', 1434383934884L)])), (u'b', u'b'), (u'c', u'c'), (u'd', u'd')])

The timestamp is not parsed out correctly. Any suggestions on how to do that correctly? (Pickle may be a direction but I am seeking other solutions first).

Answer 1

You can define your own decoder which will handle both datetime and OrderedDict and use it in object_pairs_hook. For convenience and testing I also defined my own encoder but you are free to use the one you have already.

#!/usr/bin/env python3

import json
import datetime
from collections import OrderedDict

# Test dictionary
a = OrderedDict([('a', datetime.datetime(2015, 6, 15, 15, 58, 54, 884000)),
                 ('b', 'b'), ('c', 'c'), ('d', 'd')])
print(a)

# Encoder for datetime
def encoder(obj):
    if type(obj) is datetime.datetime:
        return {'$date$': obj.timestamp()}
    raise TypeError

# Encode
s = json.dumps(a, default=encoder)
print("JSON:", s)

# Decoder for OrderedDict and datetime
def decoder(obj):
    if len(obj) == 1 and len(obj[0]) == 2 and obj[0][0] == '$date$':
        return datetime.datetime.fromtimestamp(obj[0][1])
    else:
        return OrderedDict(obj)

# Decode
b = json.loads(s, object_pairs_hook=decoder)
print(b)

# Compare
print("Comparing:", a == b)

This will print:

OrderedDict([('a', datetime.datetime(2015, 6, 15, 15, 58, 54, 884000)), ('b', 'b'), ('c', 'c'), ('d', 'd')])
JSON: {"a": {"$date$": 1434409134.884}, "b": "b", "c": "c", "d": "d"}
OrderedDict([('a', datetime.datetime(2015, 6, 15, 15, 58, 54, 884000)), ('b', 'b'), ('c', 'c'), ('d', 'd')])
Comparing: True

Answer 2

Why don't you encode/decode the datetime object directly?

import datetime as dt
import json
from collections import OrderedDict

datetime_encoding = '%Y-%m-%d %H:%M.%S %f'

od = OrderedDict([('a', dt.datetime(2015, 6, 15, 15, 58, 54, 884000).strftime(datetime_encoding)), ('b', 'b'), ('c', 'c'), ('d', 'd')])

x = json.dumps(od)

od_new = json.loads(x)
od_new['a'] = dt.datetime.strptime(od_new['a'], datetime_encoding)

>>> od_new
{u'a': datetime.datetime(2015, 6, 15, 15, 58, 54, 884000),
 u'b': u'b',
 u'c': u'c',
 u'd': u'd'}