Behaviour of Python's "yield"

Question

I'm reading about the yield keyword in python, and trying to understand running this sample:

def countfrom(n):
    while True:
        print "before yield"
        yield n
        n += 1
        print "after yield"

for i in countfrom(10):
    print "enter for loop"
    if i <= 20:
        print i
    else:
        break

The output is:

before yield
enter for loop
10
after yield
before yield
enter for loop
11
after yield
before yield
enter for loop
12
after yield
before yield
enter for loop
13
after yield
before yield
enter for loop
14
after yield
before yield
enter for loop
15
after yield
before yield
enter for loop
16
after yield
before yield
enter for loop
17
after yield
before yield
enter for loop
18
after yield
before yield
enter for loop
19
after yield
before yield
enter for loop
20
after yield
before yield
enter for loop

It looks like the yield will return the specified value, and will continue runnning the function till the end (in a parallel thread, maybe). Is my understand correct?

If you could answer this without mentioning "generators", I would be thankful, because I'm trying to understand one at a time.

Answer 1

You can think of it as if the function which yields simply "pauses" when it comes across the yield. The next time you call it, it will resume after the yield keeping the state that it was in when it left.

Answer 2

No, there is only a single thread.

Each iteration of the for loop runs your countFrom function until it yields something, or returns. After the yield, the body of the for loop runs again and then, when a new iteration starts, the countFrom function picks up exactly where it left off and runs again until it yields (or returns).

This modified version of your example will helpfully make it clearer what path execution takes.

def countfrom(n):
    while n <= 12:
        print "before yield, n = ", n
        yield n
        n += 1
        print "after yield, n = ", n

for i in countfrom(10):
    print "enter for loop, i = ", i
    print i
    print "end of for loop iteration, i = ", i

Output

before yield, n =  10
enter for loop, i =  10
10
end of for loop iteration, i =  10
after yield, n =  11
before yield, n =  11
enter for loop, i =  11
11
end of for loop iteration, i =  11
after yield, n =  12
before yield, n =  12
enter for loop, i =  12
12
end of for loop iteration, i =  12
after yield, n =  13

Answer 3

..you cannot explain the meaning of the yield statement without mentioning generators; it would be like trying to explain what a stone is without mentioning rock. That is: the yield statement is the one responsible to transform a normal function into a generator.

While you find it well documented here: http://docs.python.org/reference/simple_stmts.html#the-yield-statement

..the brief explaination of it is:

• When a function using the yield statement is called, it returns a "generator iterator", having a .next() method (the standard for iterable objects)
• Each time the .next() method of the generator is called (eg. by iterating the object with a for loop), the function is called until the first yield is encountered. Then the function execution is paused and a value is passed as return value of the .next() method.
• The next time .next() is called, the function execution is resumed until the next yield, etc. until the function returns something.

Some advantages in doing this are:

• less memory usage since memory is allocated just for the currently yielded value, not the whole list of returned values (as it would be by returning a list of values)
• "realtime" results return, as they are produced can be passed to the caller without waiting for the generation end (i used that to return output from a running process)