Python for loop skipping every other loop?

Question

I have a weird problem. Does anyone see anything wrong with my code?

for x in questions:
    forms.append((SectionForm(request.POST, prefix=str(x.id)),x))
    print "Appended " + str(x)
for (form, question) in forms:
    print "Testing " + str(question)
    if form.is_valid():
        forms.remove((form,question))
        print "Deleted " + str(question)
        a = form.save(commit=False)
        a.audit = audit
        a.save()                
    else:
        flag_error = True

Results in:

Appended Question 50
Appended Question 51
Appended Question 52
Testing Question 50
Deleted Question 50
Testing Question 52
Deleted Question 52

It seems to skip question 51. It gets appended to the list, but the for loop skips it. Any ideas?

Answer 1

You are modifying the contents of the object forms that you are iterating over, when you say:

forms.remove((form,question))

According to the Python documentation of the for statement, this is not safe (the emphasis is mine):

The for statement in Python differs a bit from what you may be used to in C or Pascal. Rather than always iterating over an arithmetic progression of numbers (like in Pascal), or giving the user the ability to define both the iteration step and halting condition (as C), Python’s for statement iterates over the items of any sequence (a list or a string), in the order that they appear in the sequence.

It is not safe to modify the sequence being iterated over in the loop (this can only happen for mutable sequence types, such as lists). If you need to modify the list you are iterating over (for example, to duplicate selected items) you must iterate over a copy. The slice notation makes this particularly convenient:

for x in a[:]: # make a slice copy of the entire list
...    if len(x) > 6: a.insert(0, x)

See also this paragraph from the Python Language Reference which explains exactly what is going on:

There is a subtlety when the sequence is being modified by the loop (this can only occur for mutable sequences, i.e. lists). An internal counter is used to keep track of which item is used next, and this is incremented on each iteration. When this counter has reached the length of the sequence the loop terminates. This means that if the suite deletes the current (or a previous) item from the sequence, the next item will be skipped (since it gets the index of the current item which has already been treated). Likewise, if the suite inserts an item in the sequence before the current item, the current item will be treated again the next time through the loop.

There are a lot of solutions. You can follow their advice and create a copy. Another possibility is to create a new list as a result of your second for loop, instead of modifying forms directly. The choice is up to you...

Answer 2

You are removing objects from forms while iterating over it. This is supposed to lead to the behaviour you are seeing ( http://docs.python.org/reference/compound_stmts.html#the-for-statement ).

The solution is to either iterate over a copy of that list, or add the forms for removal to a separate collection, then perform the removal afterwards.