Accessing the List Being Generated by List Comprehension [duplicate]

Question

So here a trivial example that is probably better executed some other way.

Here is the regular for-loop version:

lst1 = ['abc', 'abc', 'cde', 'cde']
lst2 = []
for i in lst1:
    if i not in lst2:
        lst2.append(i)

And the non-working list comprehension approximation:

lst2 = [i for i in lst1 if i not in lst2]
# NameError: name 'lst2' is not defined

So the question: is it possible to access the list being produced by a list comprehension as is it is being made?

Answer 1

No.

But if order is important, you want this answer from another question:

>>> from collections import OrderedDict
>>> list(OrderedDict.fromkeys(['abc', 'abc', 'cde', 'cde']))
['abc', 'cde']

Answer 2

TL;DR: There's no easy way to do a recursive list comprehension.

Why? It's because when the interpreter reachs this line, it'll first evaluate the right side of the operation (the list comprehension) and try to construct the list. After the construction it'll affect the list created to lst2. BUT, when you're trying to construct the list, you're calling lst2 which isn't defined yet.

You can look at the bytecode generated:

>>> def test(lst1):
...     lst2 = [i for i in lst1 if i not in lst2]
...
>>> dis.dis(test)
  2           0 BUILD_LIST               0
              3 LOAD_FAST                0 (lst1)
              6 GET_ITER
        >>    7 FOR_ITER                24 (to 34)    # the list comprehension is converted into a for loop
             10 STORE_FAST               1 (i)
             13 LOAD_FAST                1 (i)
             16 LOAD_FAST                2 (lst2)     # try to load lst2, which doesn't exist yet
             19 COMPARE_OP               7 (not in)
             22 POP_JUMP_IF_FALSE        7
             25 LOAD_FAST                1 (i)
             28 LIST_APPEND              2
             31 JUMP_ABSOLUTE            7
        >>   34 STORE_FAST               2 (lst2)
             37 LOAD_CONST               0 (None)
             40 RETURN_VALUE

Solution: What you want to do is to define a set:

>>> lst1 = ['abc', 'abc', 'cde', 'cde']
>>> set(lst1)
set(['cde', 'abc'])

(I hope you doesn't matter about the order of elements :-) ) If the order matters:

>>> tmp = set()  # create a set of already added elements
>>> [x for x in lst1 if x not in tmp and not tmp.add(x)]
['abc', 'cde']