Best method for changing a list while iterating over it [duplicate]

Question

I have several instances in a python script (v2.6) where I need to modify a list in-place. I need to pop values from the list in response to interactive input from the user and would like to know the cleanest method of doing this. Currently I have the very dirty solutions of a) setting items in the list that I want to remove to False and removing them with a filter or list comprehension or b) generating an entirely new list while going through the loop, which seems to be needlessly adding variables to the namespace and taking up memory.

An example of this problem is as follows:

for i, folder in enumerate(to_run_folders):
    if get_size(folder) < byte_threshold:
        ans = raw_input(('The folder {0}/ is less than {1}MB.' + \
                    ' Would you like to exclude it from' + \
                    ' compression? ').format(folder, megabyte_threshold))
        if 'y' in ans.strip().lower():
            to_run_folders.pop(i)

I would like to look at each folder in the list. If the current folder is less than a certain size, I want to ask the user if they want to exclude it. If they do, pop the folder from the list.

The problem with this routine is that if I iterate over the list, I get unexpected behavior and early termination. If I iterate over a copy by slicing, pop doesn't pull the right value because the indices are shifted and the problem compounds as more items are popped. I have a need for dynamic list adjustment of this kind in other areas of my script as well. Is there any clean method for this kind of functionality?

Answer 1

You can loop over the list backwards, or use a view object.

See https://stackoverflow.com/a/181062/711085 for how to loop over a list backwards. Basically use reversed(yourList) (which happens creates a view object which visits backwards).

If you require indexing, you could do reversed(enumerate(yourList)), but that would effectively create a temporary list in memory because enumerate would need to run before reversed could kick in. You will need to either do index manipulation, or to do this:

for i in xrange(len(yourList)-1, -1, -1):
    item = yourList[i]
    ...

Even cleaner: reversed is aware of range, so you can do this in python3, or in python2 if you use xrange instead:

for i in reversed(range(len(yourList))):  
    item = yourList[i]
    ...

(proof: you can do next(reversed(range(10**10))), but this will crash your computer if using python2)

Answer 2

You can loop over it backwards

Backwards:

x = range(10)
l = len(x)-1  # max index

for i, v in enumerate(reversed(x)):
    if v % 2:
        x.pop(l-i)  # l-1 is the forward index