Menu
I am producing flat lists with 10^6 to 10^7 Real numbers, and some of them are repeating.
I need to delete the repeating instances, keeping the first occurrence only, and without modifying the list order.
The key here is efficiency, as I have a lot of lists to process.
Example (fake):
Input:
{.8, .3 , .8, .5, .3, .6}
Desired Output
{.8, .3, .5, .6}
Aside note
Deleting repeating elements with Union (without preserving order) gives in my poor man's laptop:
DiscretePlot[a = RandomReal[10, i]; First@Timing@Union@a, {i, 10^6 Range@10}]
377
If you want to preserve the order while you remove duplicate elements from List in Python, you can use the OrderedDict class from the collections module. More specifically, we can use OrderedDict. fromkeys(list) to obtain a dictionary having duplicate elements removed, while still maintaining order.
Remove duplicates from list using Set. To remove the duplicates from a list, you can make use of the built-in function set(). The specialty of set() method is that it returns distinct elements.
The OrderedDict. fromkeys() method eliminates the duplicates from the list because the dictionary in Python cannot have duplicate keys. This is the fastest method in python to remove duplicates values from a list.
You want DeleteDuplicates, which preserves list order:
In[13]:= DeleteDuplicates[{.8, .3, .8, .5, .3, .6}]
Out[13]= {0.8, 0.3, 0.5, 0.6}
It was added in Mathematica 7.0.
171
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
