Extract all keys from a list of dictionaries

Question

I'm trying to get a list of all keys in a list of dictionaries in order to fill out the fieldnames argument for csv.DictWriter.

previously, I had something like this:

[ {"name": "Tom", "age": 10}, {"name": "Mark", "age": 5}, {"name": "Pam", "age": 7} ] 

and I was using fieldnames = list[0].keys() to take the first dictionary in the list and extract its keys.

Now I have something like this where one of the dictionaries has more key:value pairs than the others (could be any of the results). The new keys are added dynamically based on information coming from an API so they may or may not occur in each dictionary and I don't know in advance how many new keys there will be.

[ {"name": "Tom", "age": 10}, {"name": "Mark", "age": 5, "height":4}, {"name": "Pam", "age": 7} ] 

I can't just use fieldnames = list[1].keys() since it isn't necessarily the second element that will have extra keys.

A simple solution would be to find the dictionary with the greatest number of keys and use it for the fieldnames, but that won't work if you have an example like this:

[ {"name": "Tom", "age": 10}, {"name": "Mark", "age": 5, "height":4}, {"name": "Pam", "age": 7, "weight":90} ] 

where both the second and third dictionary have 3 keys but the end result should really be the list ["name", "age", "height", "weight"]

Answer 1

all_keys = set().union(*(d.keys() for d in mylist)) 

Edit: have to unpack the list. Now fixed.

Answer 2

Your data:

>>> LoD [{'age': 10, 'name': 'Tom'},   {'age': 5, 'name': 'Mark', 'height': 4},   {'age': 7, 'name': 'Pam', 'weight': 90}] 

This set comprehension will do it:

>>> {k for d in LoD for k in d.keys()} {'age', 'name', 'weight', 'height'} 

It works this way. First, create a list of lists of the dict keys:

>>> [list(d.keys()) for d in LoD] [['age', 'name'], ['age', 'name', 'height'], ['age', 'name', 'weight']] 

Then create a flattened version of this list of lists:

>>> [i for s in [d.keys() for d in LoD] for i in s] ['age', 'name', 'age', 'name', 'height', 'age', 'name', 'weight'] 

And create a set to eliminate duplicates:

>>> set([i for s in [d.keys() for d in LoD] for i in s]) {'age', 'name', 'weight', 'height'} 

Which can be simplified to:

{k for d in LoD for k in d.keys()}