Suppose I have this dictionary:
items = {1: {'title': u'testing123', 'description': u'testing456'},
2: {'description': u'testing123', 'description': u'testing456'},
3: {'description': u'testing123', 'description': u'testing456'},
4: {'description': u'testing123', 'description': u'testing456'},
5: {'description': u'testing123', 'description': u'testing456'},
6: {'description': u'somethingelse', 'description': u'somethingelse'}}
I want to filter out the duplicate values, so that in the end I'd get
{1: {'title': u'testing123', 'description': u'testing456'}, 6: {'title': u'something', 'description': u'somethingelse'}}
I wrote this code:
dic = {}
for key, value in items.items():
if not set(value.values()).issubset(set(dic.values())):
dic[key] = value
however I get the error message TypeError: unhashable type: 'dict'. I am not sure why this happens and how to fix it.
This is inspired by another question and my failed attempt to solve it.
dic.values() return list of dict
>>> for key, value in items.items():
... print dic.values()
...
[{'description': u'testing456', 'title': u'testing123'}]
[{'description': u'testing456', 'title': u'testing123'}]
[{'description': u'testing456', 'title': u'testing123'}]
[{'description': u'testing456', 'title': u'testing123'}]
[{'description': u'testing456', 'title': u'testing123'}]
[{'description': u'testing456', 'title': u'testing123'}]
>>>
So, you can't apply set on dict as dict is not hashable.
Btw you can fix it by:
>>> dic = {}
>>> for key, value in items.items():
... if not set(value.values()).issubset(set(sum([x.values() for x in dic.values()],[]))):
... dic[key] = value
...
>>> dic
{1: {'description': u'testing456', 'title': u'testing123'}, 6: {'description': u'somethingelse', 'title': u'somethingelse'}}
>>>
For python > 3.x
if not set(value.values()).issubset(set(sum([list(x.values()) for x in list(dic.values())],[]))):
Edit: If you must use a set, as others have noted you have to use a hashable object like a tuple:
unique_items = set()
for k, v in items.items():
sorted_v = tuple(sorted((k2, v2) for k2, v2 in v.items()))
unique_items.add(sorted_v)
unique_items = dict(unique_items)
gives for unique_items:
{1: {'description': u'testing456', 'title': u'testing123'},
6: {'description': u'somethingelse', 'title': u'somethingelse'}}
If items is not huge (or, at least if the output dictionary isn't expected to be huge):
items = {1: {'title': u'testing123', 'description': u'testing456'},
2: {'title': u'testing123', 'description': u'testing456'},
3: {'title': u'testing123', 'description': u'testing456'},
4: {'title': u'testing123', 'description': u'testing456'},
5: {'title': u'testing123', 'description': u'testing456'},
6: {'title': u'somethingelse', 'description': u'somethingelse'}}
unique_items = {}
for k, v in items.items():
if v not in unique_items.values():
unique_items[k] = v
(assuming the first key in your dictionary example should have been title). But you can't predict what the keys to this dictionary will be if duplicates do exist in items.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With