Python - Filter list of dictionaries based on multiple keys

Question

Say I have the dictionary:

myDict = [{'first': 'James', 'middle': 'Smith', 'last': 'Joule'}, 
        {'first': 'James', 'middle': 'Johnson', 'last': 'Watt'},
        {'first': 'Christian', 'middle': 'Edward', 'last': 'Doppler'}
        {'first': 'Robert', 'last': 'Antonio'}]

And I have a list called keys:

keys = ["middle", "last"]

I want to filter myDict based on each value in keys, which would result in

filteredDict = [{'middle': 'Smith', 'last': 'Joule'},
              {'middle': 'Johnson', 'last': 'Watt'},
              {'middle': 'Edward', 'last': 'Doppler'},
              {'last': 'Antonio'}]

As seen in the list dictionary in myDict list, the dictionary DOESNT have to have both keys in order to be placed into filteredDict. Is there an easy way to do this with dictionary comprehension in Python?

Answer 1

With list comprehensions:

myDict = [{'first': 'James', 'middle': 'Smith', 'last': 'Joule'}, 
      {'first': 'James', 'middle': 'Johnson', 'last': 'Watt'},
      {'first': 'Christian', 'middle': 'Edward', 'last': 'Doppler'},
      {'first': 'Robert', 'last': 'Antonio'}]

keys = {"middle", "last"}

l = [{k:v for k, v in i.items() if k in keys} for i in myDict]

But you can also use map for this:

myDict = [{'first': 'James', 'middle': 'Smith', 'last': 'Joule'}, 
      {'first': 'James', 'middle': 'Johnson', 'last': 'Watt'},
      {'first': 'Christian', 'middle': 'Edward', 'last': 'Doppler'},
      {'first': 'Robert', 'last': 'Antonio'}]

keys = {"middle", "last"}

l = list(map(lambda x: {k:v for k, v in x.items() if k in keys}, myDict))
print(l)

output:

[{'last': 'Joule', 'middle': 'Smith'}, {'last': 'Watt', 'middle': 'Johnson'}, {'last': 'Doppler', 'middle': 'Edward'}, {'last': 'Antonio'}]

Answer 2

Use neverwalkaloner's answer if you are just doing this once. But, if you find yourself manipulating lists of dictionaries often, I've written a free library called PLOD that streamlines much of this.

>>> from PLOD import PLOD
>>> l = PLOD(myDict).dropKey("middle").returnList()
>>> l
[{'last': 'Joule', 'first': 'James'}, {'last': 'Watt', 'first': 'James'}, {'last': 'Doppler', 'first': 'Christian'}, {'last': 'Antonio', 'first': 'Robert'}]
>>> print(PLOD(l).returnString())
[
    {first: 'James'    , last: 'Joule'  },
    {first: 'James'    , last: 'Watt'   },
    {first: 'Christian', last: 'Doppler'},
    {first: 'Robert'   , last: 'Antonio'}
]
>>> 

The library is on PyPi: https://pypi.python.org/pypi/PLOD

To more universally do what you want, I'd need to add a new class method. Perhaps .filterKeys. Perhaps I'll do that in version 1.8. It would then go:

>>> l = PLOD(myDict).filterKeys(['first', 'last']).returnList()

Hmmm...

BTW, the lib supports Python 2.7.x right now. Still working on the 3.5.x release.