Filter list of dictionaries with higher key value remove duplicate dictionaries

Question

I have list of dictionaries like:

sel_list = [{'a': 8}, {'a': 4}, {'a': 4}, {'b': 8}, {'b': 9}]

I want to remove duplicate dictionaries and if more than one dictionaries have same key but different values then choose that dictionary with higher value.

Like :

sel_list = [{'a': 8}, {'b': 9}]

I have tried:

[i for n, i in enumerate(sel_list) if i not in sel_list[n + 1:]]

its results in:

[{'a': 8}, {'a': 4}, {'b': 8}, {'b': 9}]

What I can do to achieve my results?

Answer 1

We can do this by constructing a dictionary that "folds" the values by picking the maximum each time. Like:

dummy = object()
maximums = {}
for subd in sel_list:
    for k, v in subd.items():
        cur = maximums.get(k, dummy)
        if cur is dummy or v > cur:
            maximums[k] = v
result = [{k: v} for k, v in maximums.items()]

We thus iterate over the key-value pairs of the dictionaries in the list, and each time update the maximums dictionary in case the key does not yet exists, or the current value is less.

After this iteration step, we produce a list of dictionaries with the maximum key-value pairs.

This approach works on all types that can be ordered (numbers, strings, etc.), and the keys should be hashable, but this assumption holds since in the list of dictionaries, the keys are already hashed.

Furthermore it works rather robust in the sense that it will ignore empty dictionaries, and will process a dictionary with multiple key-value pairs as well, by seeing these as independent key-value pairs.

You can also decide to work with maximums directly: a dictionary that contains all the keys in your original list, and associates these with the maximum value seen in the list.