How to join 2 lists of dicts in python?

Question

I have 2 lists like this:

l1 = [{'a': 1, 'b': 2, 'c': 3, 'd': 4}, {'a': 5, 'b': 6, 'c': 7, 'd': 8}]

l2 = [{'a': 5, 'b': 6, 'e': 100}, {'a': 1, 'b': 2, 'e': 101}]

and I want to obtain a list l3, which is a join of l1 and l2 where values of 'a' and 'b' are equal in both l1 and l2 i.e.

l3 = [{'a': 1, 'b: 2, 'c': 3, 'd': 4, 'e': 101}, {'a': 5, 'b: 6, 'c': 7, 'd': 8, 'e': 100}]

How can I do this?

Answer 1

You should accumulate the results in a dictionary. You should use the values of 'a' and 'b' to form a key of this dictionary

Here, I have used a defaultdict to accumulate the entries

l1 = [{'a': 1, 'b': 2, 'c': 3, 'd': 4}, {'a': 5, 'b': 6, 'c': 7, 'd': 8}]
l2 = [{'a': 5, 'b': 6, 'e': 100}, {'a': 1, 'b': 2, 'e': 101}]

from collections import defaultdict
D = defaultdict(dict)
for lst in l1, l2:
    for item in lst:
        key = item['a'], item['b']
        D[key].update(item)

l3 = D.values()
print l3

output:

[{'a': 1, 'c': 3, 'b': 2, 'e': 101, 'd': 4}, {'a': 5, 'c': 7, 'b': 6, 'e': 100, 'd': 8}]

Answer 2

Simple list operations would do the thing for you as well:

l1 = [{'a': 1, 'b': 2, 'c': 3, 'd': 4}, {'a': 5, 'b': 6, 'c': 7, 'd': 8}]
l2 = [{'a': 5, 'b': 6, 'e': 100}, {'a': 1, 'b': 2, 'e': 101}]
l3 = []

for i in range(len(l1)):
    for j in range(len(l2)):
        if l1[i]['a'] == l2[j]['a'] and l1[i]['b'] == l2[j]['b']:
            l3.append(dict(l1[i]))
            l3[i].update(l2[j])

Answer 3

My approach is to sort the the combined list by the key, which is keys a + b. After that, for each group of dictionaries with similar key, combine them:

from itertools import groupby

def ab_key(dic):
    return dic['a'], dic['b']

def combine_lists_of_dicts(list_of_dic1, list_of_dic2, keyfunc):
    for key, dic_of_same_key in groupby(sorted(list_of_dic1 + list_of_dic2, key=keyfunc), keyfunc):
        combined_dic = {}
        for dic in dic_of_same_key:
            combined_dic.update(dic)
        yield combined_dic

l1 = [{'a': 1, 'b': 2, 'c': 3, 'd': 4}, {'a': 5, 'b': 6, 'c': 7, 'd': 8}]
l2 = [{'a': 5, 'b': 6, 'e': 100}, {'a': 1, 'b': 2, 'e': 101}]

for dic in combine_lists_of_dicts(l1, l2, ab_key):
    print dic

Discussion

• The function ab_key returns a tuple of value for key a and b, used for sorting a groupping
• The groupby function groups all the dictionaries with similar keys together
• This solution is less efficient than that of John La Rooy, but should work fine for small lists