Navigating Multi-Dimensional JSON arrays in Python

Question

I'm trying to figure out how to query a JSON array in Python. Could someone show me how to do a simple search and print through a fairly complex array please?

The example I'm using is here: http://eu.battle.net/api/wow/realm/status

I'd like to see, for example, how to find the 'Silvermoon' server, and print say its 'population', then the 'controlling-faction' within the 'Wintergrasp' array.

The array snippet currently looks like this:

{"type":"pve","population":"high","queue":false,"wintergrasp":{"area":1,"controlling-faction":0,"status":0,"next":1382350068792},"tol-barad":{"area":21,"controlling-faction":0,"status":0,"next":1382349141932},"status":true,"name":"Silvermoon","slug":"silvermoon","battlegroup":"Cyclone / Wirbelsturm","locale":"en_GB","timezone":"Europe/Paris"}

At the moment I can access the main array, but don't seem to be able to access sub-arrays without copying the whole thing to another new variable which seems wasteful. I'd like to be able to do something like

import urllib2
import json

req = urllib2.Request("http://eu.battle.net/api/wow/realm/status", None, {})
opener = urllib2.build_opener()
f = opener.open(req)

x = json.load(f)  # open the file and read into a variable

# search and find the Silvermoon server
silvermoon_array = ????

# print the population
print silvermoon_array.????

# access the Wintergrasp sub-array
wintergrasp_sub = ????
print wintergrasp_sub.????  # print the controlling-faction variable

This would really help me get to grips with how to access other things.

Answer 1

Python's interactive mode is a great way to progressively explore structured data. It's easy to find how to access, say, the silvermoon server data:

>>> data=json.load(urllib2.urlopen("http://eu.battle.net/api/wow/realm/status"))
>>> type(data)
<type 'dict'>
>>> data.keys()
[u'realms']
>>> type(data['realms'])
<type 'list'>
>>> type(data['realms'][0])
<type 'dict'>
>>> data['realms'][0].keys()
[u'status', u'wintergrasp', u'battlegroup', u'name', u'tol-barad', u'locale', u'queue', u'timezone', u'type', u'slug', u'population']
>>> data['realms'][0]['name']
u'Aegwynn'
>>> [realm['name'] for realm in data['realms']].index('Silvermoon')
212
>>> silvermoon= data['realms'][212]
>>> silvermoon['population']
u'high'
>>> type(silvermoon['wintergrasp'])
<type 'dict'>
>>> silvermoon['wintergrasp'].keys()
[u'status', u'next', u'controlling-faction', u'area']
>>> silvermoon['wintergrasp']['controlling-faction']
>>> silvermoon['population']
u'high'

If you don't know about them yet, you should read up on dictionary.keys, list.index and list comprehensions to understand what's going on.

After figuring out the structure of the data, you can finally rewrite the data access to be a bit more readable and efficient:

realms= data['realms']
realm_from_name= dict( [(realm['name'], realm) for realm in realms])
print realm_from_name['Silvermoon']['population']
print realm_from_name['Silvermoon']['wintergrasp']['controlling-faction']

As to copying the array to another variable being wasteful, you should know that python passes value by reference. That means that there's no copying involved when you assign something to a new variable. Here's a simple explanation of passing by value vs passing by reference

Finally, you seem to be excessively worried with performance. Python's philosophy is get it right first, optimize later. When you have this working correctly and if you need better performance, optimize it, if it's worth the time.

Answer 2

This does what you want:

# -*- coding: utf-8 -*-
import json
import urllib2

def searchListOfDicts(listOfDicts, attr, value):
    """
    Loops through a list of dictionaries and returns matching attribute value pair
    You can also pass it slug, silvermoon or type, pve
    returns a list containing all matching dictionaries 
    """
    matches = [record for record in listOfDicts if attr in record and record[attr] == value]
    return matches

def myjsonDict():
    """
    Opens url, grabs json and puts it inside a dictionary
    """
    req = urllib2.Request("http://eu.battle.net/api/wow/realm/status", None, {})
    opener = urllib2.build_opener()
    f = opener.open(req)
    json_dict = json.load(f)
    return json_dict

jsonDict = myjsonDict()
#we want to search inside realms list
silverMoonServers = searchListOfDicts(jsonDict["realms"], "name", "Silvermoon")
#access first dictionary that matched "name, Silvermoon" query
print silverMoonServers[0]
print silverMoonServers[0]["wintergrasp"]
print silverMoonServers[0]["wintergrasp"]["controlling-faction"]