Python Remove SOME duplicates from a list while maintaining order?

Question

I want to remove certain duplicates in my python list. I know there are ways to remove all duplicates, but I wanted to remove only consecutive duplicates, while maintaining the list order.

For example, I have a list such as the following:

list1 = [a,a,b,b,c,c,f,f,d,d,e,e,f,f,g,g,c,c]

However, I want to remove the duplicates, and maintain order, but still keep the 2 c's and 2 f's, such as this:

wantedList = [a,b,c,f,d,e,f,g,c]

So far, I have this:

z = 0
j=0
list2=[]
for i in list1:
    if i == "c":
        z = z+1
        if (z==1):
            list2.append(i)
        if (z==2):
            list2.append(i)
        else:
            pass
    elif i == "f":
        j = j+1
        if (j==1):
            list2.append(i)
        if (j==2):
            list2.append(i)
        else:
            pass
    else:
        if i not in list2:
            list2.append(i)  

However, this method gives me something like:

wantedList = [a,b,c,c,d,e,f,f,g]

Thus, not maintaining the order.

Any ideas would be appreciated! Thanks!

Answer 1

Not completely sure if c and f are special cases, or if you want to compress consecutive duplicates only. If it is the latter, you can use itertools.groupby():

>>> import itertools
>>> list1
['a', 'a', 'b', 'b', 'c', 'c', 'f', 'f', 'd', 'd', 'e', 'e', 'f', 'f', 'g', 'g', 'c', 'c']
>>> [k for k, g in itertools.groupby(list1)]
['a', 'b', 'c', 'f', 'd', 'e', 'f', 'g', 'c']

Answer 2

To remove consecutive duplicates from a list, you can use the following generator function:

def remove_consecutive_duplicates(a):
    last = None
    for x in a:
        if x != last:
            yield x
        last = x

With your data, this gives:

>>> list1 = ['a','a','b','b','c','c','f','f','d','d','e','e','f','f','g','g','c','c']
>>> list(remove_consecutive_duplicates(list1))
['a', 'b', 'c', 'f', 'd', 'e', 'f', 'g', 'c']