Menu
If I have a string list ,
a = ["asd","def","ase","dfg","asd","def","dfg"]
how can I remove the duplicates from the list?
397
You can remove duplicates from a Python using the dict. fromkeys(), which generates a dictionary that removes any duplicate values. You can also convert a list to a set. You must convert the dictionary or set back into a list to see a list whose duplicates have been removed.
We use file handling methods in python to remove duplicate lines in python text file or function. The text file or function has to be in the same directory as the python program file. Following code is one way of removing duplicates in a text file bar. txt and the output is stored in foo.
In addition, we can say itertools groupby is one of the best methods to remove Duplicates from the Python list. As it uses the builtin module, so by default it’s speed will be higher than the above-mentioned methods to remove duplication in Python list.
Just remove the iteration part. Also, an easy way to remove duplicates, if you are not interested in the order of the elements is to use set. So, in your case you can initialize lst as - lst=set () . And then use lst.add () element, you would not even need to do a check whether the element already exists or not.
A List with Duplicates. mylist = ["a", "b", "a", "c", "c"] mylist = list (dict.fromkeys (mylist)) print(mylist) Create a dictionary, using the List items as keys. This will automatically remove any duplicates because dictionaries cannot have duplicate keys.
This is what we want to achieve. So we are not removing all duplicated elements. We are only eliminating consecutive duplicate elements. Let’s check out the 3 different ways.
Convert to a set:
a = set(a)
Or optionally back to a list:
a = list(set(a))
Note that this doesn't preserve order. If you want to preserve order:
seen = set()
result = []
for item in a:
if item not in seen:
seen.add(item)
result.append(item)
See it working online: ideone
181
Use the set type to remove duplicates
a = list(set(a))
def getUniqueItems(iterable):
result = []
for item in iterable:
if item not in result:
result.append(item)
return result
print (''.join(getUniqueItems(list('apple'))))
P.S. Same thing like one of the answers here but a little change, set is not really required !
6
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
