Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

python remove duplicates from 2 lists

I am trying to remove duplicates from 2 lists. so I wrote this function:

a = ["abc", "def", "ijk", "lmn", "opq", "rst", "xyz"]

b = ["ijk", "lmn", "opq", "rst", "123", "456", ]

for i in b:
    if i in a:
        print "found " + i
        b.remove(i)

print b

But I find that the matching items following a matched item does not get remove.

I get result like this:

found ijk
found opq
['lmn', 'rst', '123', '456']

but i expect result like this:

['123', '456']

How can I fix my function to do what I want?

Thank you.

like image 450
michael Avatar asked Aug 12 '13 19:08

michael


People also ask

How do you remove common elements from two lists?

To remove the common elements from two lists:Use the set() class to convert the first list to a set object. Use the symmetric_difference() method to remove the common elements. The method will return all elements that are in exactly one of the lists.

How do you remove duplicates from a consecutive list in Python?

Using the groupby function, we can group the together occurring elements as one and can remove all the duplicates in succession and just let one element be in the list. This function can be used to keep the element and delete the successive elements with the use of slicing.


2 Answers

Your problem seems to be that you're changing the list you're iterating over. Iterate over a copy of the list instead.

for i in b[:]:
    if i in a:
        b.remove(i)


>>> b
['123', '456']

But, How about using a list comprehension instead?

>>> a = ["abc", "def", "ijk", "lmn", "opq", "rst", "xyz"]
>>> b = ["ijk", "lmn", "opq", "rst", "123", "456", ]
>>> [elem for elem in b if elem not in a ]
['123', '456']
like image 141
Sukrit Kalra Avatar answered Oct 09 '22 14:10

Sukrit Kalra


Here is what's going on. Suppose you have this list:

['a', 'b', 'c', 'd']

and you are looping over every element in the list. Suppose you are currently at index position 1:

['a', 'b', 'c', 'd']
       ^
       |
   index = 1

...and you remove the element at index position 1, giving you this:

['a',      'c', 'd']
       ^
       |
    index 1

After removing the item, the other items slide to the left, giving you this:

['a', 'c', 'd']
       ^
       |
    index 1

Then when the loop runs again, the loop increments the index to 2, giving you this:

['a', 'c', 'd']
            ^ 
            |
         index = 2

See how you skipped over 'c'? The lesson is: never delete an element from a list that you are looping over.

like image 45
7stud Avatar answered Oct 09 '22 12:10

7stud