JSON keys are shuffled in Python

Question

I'm parsing the JSON from the web api, and Python seems to shuffle the keys when I iterate over them.

Original JSON on screenshot (it's right original ordering. No, it's not just alphabetically sorted):

My code:

data = requests.get('http://www.dota2.com/jsfeed/heropediadata?feeds=abilitydata&l=english').json()

for key in data['abilitydata']:
    print key

And output:

tiny_avalanche
rubick_fade_bolt
doom_bringer_devour
undying_flesh_golem
...

I've also tried to do this via urllib & json.loads() — it gives the same result.

How can I can achieve the original ordering?

Answer 1

You can use an ordered dictionary and the object_pairs_hook argument of the loads method in the json package. Here is a working code example:

import json
import requests
from collections import OrderedDict

result = requests.get('http://www.dota2.com/jsfeed/heropediadata?feeds=abilitydata&l=english')
data = json.loads(result.text, object_pairs_hook = OrderedDict)

data will contain your dictionary keys in order

Answer 2

As the others have said, dict is unordered. collections.OrderedDict is a dict subclass whose keys are ordered. The problem is that json.load returns a dict directly, and we can't just throw the result in an OrderedDict, as the order of the keys is already lost information by this point.

We need a way of telling json.load to return an OrderedDict instead of a dict.This can be done by implementing a custom json.JSONDecoder which supplies an object_pairs_hook. The object_pairs_hook is given a JSON object as a list of (key, value) tuples in the order that they appear in the JSON document. It should return the translation of this object to a Python object. We'll hand this list of tuples to the initializer for collections.OrderedDict, and that should do the trick.

Here's some code:

data = """
{
    "foo": "bar",
    "a_list": [1, 2, 3],
    "another_object": {
        "c": 3,
        "a": 1,
        "b": 2
        },
    "last_key": 42
}
"""

decoder = json.JSONDecoder(object_pairs_hook=collections.OrderedDict)
result = decoder.decode(data)
print(result)

which gives:

OrderedDict([('foo', 'bar'),
             ('a_list', [1, 2, 3]),
             ('another_object', OrderedDict([('c', 3), ('a', 1), ('b', 2)])),
             ('last_key', 42)])

Lastly, you might be wondering: "why is this so much extra work?". Well, JSON isn't meant to be treated as a data structure with any fixed order. You're going against the grain by doing this.