How does a for loop evaluate its argument

Question

My question is a very simple one.

Does a for loop evaluates the argument it uses every time ?

Such as:

for i in range(300):

Does python create a list of 300 items for every iteration of this loop?

If it is, is this a way to avoid it?

lst = range(300)
for i in lst:
    #loop body

Same goes for code examples like this.

for i in reversed(lst):

for k in range(len(lst)):

Is the reverse process applied every single time, or the length calculated at every iteration? (I ask this for both python2 and python3)

If not, how does Python evaluate the changes on the iterable while iterating over it ?

Answer 1

No fear, the iterator will only be evaluated once. It ends up being roughly equivalent to code like this:

it = iter(range(300))
while True:
    try:
        i = next(it)
    except StopIteration:
        break
    ... body of loop ...

Note that it's not quite equivalent, because break will work differently. Remember that you can add an else to a for loop, but that won't work in the above code.

Answer 2

What objects are created depends on what the __iter__ method of the Iterable you are looping over returns.

Usually Python creates one Iterator when iterating over an Iterable which itself is not an Iterator. In Python2, range returns a list, which is an Iterable and has an __iter__ method which returns an Iterator.

>>> from collections import Iterable, Iterator
>>> isinstance(range(300), Iterable)
True
>>> isinstance(range(300), Iterator)
False
>>> isinstance(iter(range(300)), Iterator)
True

The for in sequence: do something syntax is basically a shorthand for doing this:

it = iter(some_iterable) # get Iterator from Iterable, if some_iterable is already an Iterator, __iter__ returns self by convention
while True:
    try:
        next_item = next(it)
        # do something with the item
    except StopIteration:
        break

Here is a demo with some print statements to clarify what's happening when using a for loop:

class CapitalIterable(object):
    'when iterated over, yields capitalized words of string initialized with'
    def __init__(self, stri):
        self.stri = stri

    def __iter__(self):
        print('__iter__ has been called')
        return CapitalIterator(self.stri)

        # instead of returning a custom CapitalIterator, we could also
        # return iter(self.stri.title().split())
        # because the built in list has an __iter__ method

class CapitalIterator(object):
    def __init__(self, stri):
        self.items = stri.title().split()
        self.index = 0

    def next(self): # python3: __next__
        print('__next__ has been called')
        try:
            item = self.items[self.index]
            self.index += 1
            return item
        except IndexError:
            raise StopIteration

    def __iter__(self):
        return self

c = CapitalIterable('The quick brown fox jumps over the lazy dog.')
for x in c:
    print(x)

Output:

__iter__ has been called
__next__ has been called
The
__next__ has been called
Quick
__next__ has been called
Brown
__next__ has been called
Fox
__next__ has been called
Jumps
__next__ has been called
Over
__next__ has been called
The
__next__ has been called
Lazy
__next__ has been called
Dog.
__next__ has been called

As you can see, __iter__ is being called only once, therefore only one Iterator object is created.