Avoiding KeyError when iterating over a list of dictionaries

Question

I have a list of dictionaries:

test = [{'first': '1'}, {'second': '2'}, {'first': '0'}, {'third': '3'}, {'fourth': '4'}]

but when I do:

stuff = [L['first'] for L in test]
print(stuff)

I get this:

Traceback (most recent call last):
  File "C:/Users/User/Desktop/test_run.py", line 4, in <module>
    stuff = [L['first'] for L in test]
  File "C:/Users/User/Desktop/test_run.py", line 4, in <listcomp>
    stuff = [L['first'] for L in test]
KeyError: 'first'

I know I might be doing a silly mistake but any help?

Answer 1

List comprehension + if

If you want all the values, you need to check the dict has the corresponding key first:

>>> [d['first'] for d in test if 'first' in d]
['1', '0']
>>> [d['sixth'] for d in test if 'sixth' in d]
[]

Just one value

You could use next to get the value corresponding to the first occurence of 'first', if you're sure they're at least one dict with a 'first' value:

>>> test = [{'first': '1'}, {'second': '2'}, {'first': '0'}, {'third': '3'}, {'fourth': '4'}]
>>> next(d['first'] for d in test if 'first' in d)
'1'

It raises a StopIteration otherwise:

>>> next(d['sixth'] for d in test if 'sixth' in d)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration

Alternative data format

Finally, if you do this operation often, it might be interesting to change the format slightly:

from collections import defaultdict
data = defaultdict(list)

test = [{'first': '1'}, {'second': '2'}, {'first': '0'}, {'third': '3'}, {'fourth': '4'}]

for d in test:
    for k in d:
        data[k].append(d[k])

print(data)
# defaultdict(<type 'list'>, {'second': ['2'], 'fourth': ['4'], 'third': ['3'], 'first': ['1', '0']})
print(data['first'])
# ['1', '0']
print(data['sixth'])
# []

The for loop is only needed once, the lookup is very fast afterwards.

Answer 2

This can be solved with comprehensions but personally I would simply use a plain for loop - mostly because that way it doesn't require the if condition. Wrapped as function it could also be easily reused (i.e. for other keys):

def get_key(list_of_dicts, key):
    for dct in test:
        try:
            yield dct[key]
        except KeyError:
            pass

This is a generator so you can cast it to a list, or use it wherever you would iterate over it:

>>> list(get_key(test, 'first'))
['1', '0']