Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Unique list of dicts based on keys

I have a list of dics:

     data = {}
     data['key'] = pointer_key
     data['timestamp'] = timestamp
     data['action'] = action
     data['type'] = type
     data['id'] = id

     list = [data1, data2, data3, ... ]

How can I ensure that for each data item in the list, only one such element exists for each "key"? If there are two keys as seen below, the most recent timestamp would win:

    list = [{'key':1,'timestamp':1234567890,'action':'like','type':'photo',id:245},
            {'key':2,'timestamp':2345678901,'action':'like','type':'photo',id:252},
            {'key':1,'timestamp':3456789012,'action':'like','type':'photo',id:212}]

    unique(list)

    list = [{'key':2,'timestamp':2345678901,'action':'like','type':'photo',id:252},
            {'key':1,'timestamp':3456789012,'action':'like','type':'photo',id:212}]

Thanks.

like image 933
ensnare Avatar asked Dec 06 '10 20:12

ensnare


People also ask

How do I get a list of unique values from a dictionary?

Method #1 : Using set() + values() + dictionary comprehension The combination of these methods can together help us achieve the task of getting the unique values. The values function helps us get the values of dictionary, set helps us to get the unique of them, and dictionary comprehension to iterate through the list.

Can Dicts have same keys?

In this article, we will find out whether a dictionary has duplicate keys or not in Python. The straight answer is NO. You can not have duplicate keys in a dictionary in Python.

Are Keys in dictionaries unique?

No, each key in a dictionary should be unique. You can't have two keys with the same value. Attempting to use the same key again will just overwrite the previous value stored. If a key needs to store multiple values, then the value associated with the key should be a list or another dictionary.

Are Dicts faster than lists?

A dictionary is 6.6 times faster than a list when we lookup in 100 items.


2 Answers

Here's my solution:

def uniq(list_dicts):
    return [dict(p) for p in set(tuple(i.items()) 
        for i in list_dicts)]

hope it will help somebody.

like image 146
Alex Bojko Avatar answered Sep 30 '22 13:09

Alex Bojko


I needed this, but didn't like any of the answers here. So I made this simple and performant version.

def list_of_seq_unique_by_key(seq, key):
    seen = set()
    seen_add = seen.add
    return [x for x in seq if x[key] not in seen and not seen_add(x[key])]

# Usage
# If you want most recent timestamp to win, just sort by timestamp first
list = sorted(list, key=lambda k: k['timestamp'], reverse=True)
# Remove everything with a duplicate value for key 'key'
list = list_of_seq_unique_by_key(list, 'key')
like image 25
Farzher Avatar answered Sep 30 '22 14:09

Farzher