Python merge dictionaries with custom merge function

Question

I want to merge two dictionaries A and B such that the result contains:

• All pairs from A where key is unique to A
• All pairs from B where key is unique to B
• f(valueA, valueB) where the same key exists in both A and B

For example:

def f(x, y):
    return x * y

A = {1:1, 2:3}
B = {7:3, 2:2}

C = merge(A, B)

Output:

{1:1, 7:3, 2:6}

It feels like there should be a nice one-liner to do this.

Answer 1

Use dictionary views to achieve this; the dict.viewkeys() result acts like a set and let you do intersections and symmetrical differences:

def merge(A, B, f):
    # Start with symmetric difference; keys either in A or B, but not both
    merged = {k: A.get(k, B.get(k)) for k in A.viewkeys() ^ B.viewkeys()}
    # Update with `f()` applied to the intersection
    merged.update({k: f(A[k], B[k]) for k in A.viewkeys() & B.viewkeys()})
    return merged

In Python 3, the .viewkeys() method has been renamed to .keys(), replacing the old .keys() functionality (which in Python 2 returs a list).

The above merge() method is the generic solution which works for any given f().

Demo:

>>> def f(x, y):
...     return x * y
... 
>>> A = {1:1, 2:3}
>>> B = {7:3, 2:2}
>>> merge(A, B, f)
{1: 1, 2: 6, 7: 3}
>>> merge(A, B, lambda a, b: '{} merged with {}'.format(a, b))
{1: 1, 2: '3 merged with 2', 7: 3}

Answer 2

Stealing this (A.get(k, B.get(k)) snippet from @MartijnPieters

>>> def f(x, y):
        return x * y

>>> A = {1:1, 2:3}
>>> B = {7:3, 2:2}
>>> {k: f(A[k], B[k]) if k in A and k in B else A.get(k, B.get(k))
     for k in A.viewkeys() | B.viewkeys()}
{1: 1, 2: 6, 7: 3}

Answer 3

Here's my solution code in Python 3 for the general case.

I first wrote the merge function and then extend it to the more general merge_with function, which takes a function and various number of dictionaries. Were there any duplicate keys in those dictionaries, apply the supplied function to the values whose keys are duplicate.

The merge function can be redefined using the merge_with function, as in the case of merger function. The name merger means to merge them all and keep the rightmost values, were there any duplicates. So does the mergel function, which keep the leftmost.

All the functions here — merge, merge_with, mergel, and merger — are generic in the case that they take arbitrary number of dictionary arguments. Specifically, merge_with must take as argument a function compatible with the data to which it will apply.

from functools import reduce
from operator import or_

def merge(*dicts):
    return { k: reduce(lambda d, x: x.get(k, d), dicts, None)
             for k in reduce(or_, map(lambda x: x.keys(), dicts), set()) }

def merge_with(f, *dicts):
    return { k: (lambda x: f(*x) if len(x)>1 else x[0])([ d[k] for d in dicts
                                                          if k in d ])
             for k in reduce(or_, map(lambda x: x.keys(), dicts), set()) }

mergel = lambda *dicts: merge_with(lambda *x: x[0], *dicts)

merger = lambda *dicts: merge_with(lambda *x: x[-1], *dicts)

Tests

>>> squares = { k:k*k for k in range(4) }
>>> squares
{0: 0, 1: 1, 2: 4, 3: 9}
>>> cubes = { k:k**3 for k in range(2,6) }
>>> cubes
{2: 8, 3: 27, 4: 64, 5: 125}
>>> merger(squares, cubes)
{0: 0, 1: 1, 2: 8, 3: 27, 4: 64, 5: 125}
>>> merger(cubes, squares)
{0: 0, 1: 1, 2: 4, 3: 9, 4: 64, 5: 125}
>>> mergel(squares, cubes)
{0: 0, 1: 1, 2: 4, 3: 9, 4: 64, 5: 125}
>>> mergel(cubes, squares)
{0: 0, 1: 1, 2: 8, 3: 27, 4: 64, 5: 125}
>>> merge(squares, cubes)
{0: 0, 1: 1, 2: 8, 3: 27, 4: 64, 5: 125}
>>> merge(cubes, squares)
{0: 0, 1: 1, 2: 4, 3: 9, 4: 64, 5: 125}
>>> merge_with(lambda x, y: x+y, squares, cubes)
{0: 0, 1: 1, 2: 12, 3: 36, 4: 64, 5: 125}
>>> merge_with(lambda x, y: x*y, squares, cubes)
{0: 0, 1: 1, 2: 32, 3: 243, 4: 64, 5: 125}

Update

After I wrote the above, I find there's another way to do it.

from functools import reduce

def merge(*dicts):
    return reduce(lambda d1, d2: reduce(lambda d, t:
                                        dict(list(d.items())+[t]),
                                        d2.items(), d1),
                  dicts, {})

def merge_with(f, *dicts):
    return reduce(lambda d1, d2: reduce(lambda d, t:
                                        dict(list(d.items()) +
                                             [(t[0], f(d[t[0]], t[1])
                                               if t[0] in d else
                                               t[1])]),
                                        d2.items(), d1),
                  dicts, {})

mergel = lambda *dicts: merge_with(lambda x, y: x, *dicts)
merger = lambda *dicts: merge_with(lambda x, y: y, *dicts)

Notice that the definitions for mergel and merger using merge_with have been changed with new functions as first arguments. The f function must now be binary. The tests provided above still works. Here are some more tests to show the generality of those functions.

>>> merge() == {}
True
>>> merge(squares) == squares
True
>>> merge(cubes) == cubes
True
>>> mergel() == {}
True
>>> mergel(squares) == squares
True
>>> mergel(cubes) == cubes
True
>>> merger() == {}
True
>>> merger(squares) == squares
True
>>> merger(cubes) == cubes
True
>>> merge_with(lambda x, y: x+y, squares, cubes, squares)
{0: 0, 1: 2, 2: 16, 3: 45, 4: 64, 5: 125}
>>> merge_with(lambda x, y: x*y, squares, cubes, squares)
{0: 0, 1: 1, 2: 128, 3: 2187, 4: 64, 5: 125}