Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to make lists contain only distinct element in Python? [duplicate]

Tags:

python

People also ask

How do you only get unique elements from a list in Python?

Using Python's import numpy, the unique elements in the array are also obtained. In the first step convert the list to x=numpy. array(list) and then use numpy. unique(x) function to get the unique values from the list.

How do you store unique values in Python?

By using set() method In Python the set method stores only unique values it will automatically remove duplicate values from the list when we apply the function 'set()'. In python, the set is an unordered collection of elements and we can easily remove or update value from it.

How do I get distinct items from a list?

List<int> myList = list. Distinct(). ToList();


The simplest is to convert to a set then back to a list:

my_list = list(set(my_list))

One disadvantage with this is that it won't preserve the order. You may also want to consider if a set would be a better data structure to use in the first place, instead of a list.


Modified versions of http://www.peterbe.com/plog/uniqifiers-benchmark

To preserve the order:

def f(seq): # Order preserving
  ''' Modified version of Dave Kirby solution '''
  seen = set()
  return [x for x in seq if x not in seen and not seen.add(x)]

OK, now how does it work, because it's a little bit tricky here if x not in seen and not seen.add(x):

In [1]: 0 not in [1,2,3] and not print('add')
add
Out[1]: True

Why does it return True? print (and set.add) returns nothing:

In [3]: type(seen.add(10))
Out[3]: <type 'NoneType'>

and not None == True, but:

In [2]: 1 not in [1,2,3] and not print('add')
Out[2]: False

Why does it print 'add' in [1] but not in [2]? See False and print('add'), and doesn't check the second argument, because it already knows the answer, and returns true only if both arguments are True.

More generic version, more readable, generator based, adds the ability to transform values with a function:

def f(seq, idfun=None): # Order preserving
  return list(_f(seq, idfun))

def _f(seq, idfun=None):  
  ''' Originally proposed by Andrew Dalke '''
  seen = set()
  if idfun is None:
    for x in seq:
      if x not in seen:
        seen.add(x)
        yield x
  else:
    for x in seq:
      x = idfun(x)
      if x not in seen:
        seen.add(x)
        yield x

Without order (it's faster):

def f(seq): # Not order preserving
  return list(set(seq))

one-liner and preserve order

list(OrderedDict.fromkeys([2,1,1,3]))

although you'll need

from collections import OrderedDict

To preserve the order:

l = [1, 1, 2, 2, 3]
result = list()
map(lambda x: not x in result and result.append(x), l)
result
# [1, 2, 3]

Let me explain to you by an example:

if you have Python list

>>> randomList = ["a","f", "b", "c", "d", "a", "c", "e", "d", "f", "e"]

and you want to remove duplicates from it.

>>> uniqueList = []

>>> for letter in randomList:
    if letter not in uniqueList:
        uniqueList.append(letter)

>>> uniqueList
['a', 'f', 'b', 'c', 'd', 'e']

This is how you can remove duplicates from the list.