Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

GeneratorExit in Python generator

I wrote a test program about Python generator. But I got an error that is not expected. And I don't know how to explain it. Let me show you the code:

def countdown(n):
    logging.debug("Counting down")
    while n > 0:
        try:
            yield n
        except GeneratorExit:
            logging.error("GeneratorExit")
        n -= 1


if __name__ == '__main__':
    c = countdown(10)
    logging.debug("value: %d", c.next())

I thought it should run without any problem. But the output is:

# ./test.py
[2015/06/16 04:10:49] DEBUG    - Counting down     
[2015/06/16 04:10:49] DEBUG    - value: 10 
[2015/06/16 04:10:49] ERROR    - GeneratorExit
Exception RuntimeError: 'generator ignored GeneratorExit' in <generator object countdown at 0x7f9934407640> ignored

Why is there an error at the last line. I don't know why I triggered the GeneratorExit exception. Is there something aobut generator I missed? I also typed the code into interactive python shell, and everything is OK. How can this happen?

like image 908
ruanhao Avatar asked Jun 16 '15 08:06

ruanhao


People also ask

What is GeneratorExit in Python?

If the generator function then exits gracefully, is already closed, or raises GeneratorExit (by not catching the exception), close returns to its caller. If the generator yields a value, a RuntimeError is raised. If the generator raises any other exception, it is propagated to the caller.

How do I create a custom generator in Python?

Create Generators in Python It is fairly simple to create a generator in Python. It is as easy as defining a normal function, but with a yield statement instead of a return statement. If a function contains at least one yield statement (it may contain other yield or return statements), it becomes a generator function.

Can a generator be called multiple times in Python?

Yes, generator can be used only once. but you have two generator object. Save this answer.

How does yield in Python work?

The Yield keyword in Python is similar to a return statement used for returning values or objects in Python. However, there is a slight difference. The yield statement returns a generator object to the one who calls the function which contains yield, instead of simply returning a value.


3 Answers

Suppose you have the following generator:

def gen():
    with open('important_file') as f:
        for line in f:
            yield line

and you next it once and throw it away:

g = gen()
next(g)
del g

The generator's control flow never leaves the with block, so the file doesn't get closed. To prevent this, when a generator is garbage-collected, Python calls its close method, which raises a GeneratorExit exception at the point from which the generator last yielded. This exception is intended to trigger any finally blocks or context manager __exit__s that didn't get a chance to run.

When you catch the GeneratorExit and keep going, Python sees that the generator didn't exit properly. Since that can indicate that resources weren't properly released, Python reports this as a RuntimeError.

like image 51
user2357112 supports Monica Avatar answered Oct 18 '22 07:10

user2357112 supports Monica


When the generator object is garbage-collected at the end of your program, its close() method is called, and this raises the GeneratorExit exception inside the generator. Normally this is not caught and causes the generator to exit.

Since you catch the exception and proceed to yield another value, this causes a RuntimeError. If you catch the GeneratorExit exception you need to either reraise it, or exit the function without yielding anything else.

like image 28
interjay Avatar answered Oct 18 '22 07:10

interjay


Maybe It's because your's yield value n is inside try blocks which always returns new n reference which make the last n value is garbage collected automatically. It's also state in PEP 342 :

"Add support to ensure that close() is called when a generator iterator is garbage-collected"

"Allow yield to be used in try/finally blocks, since garbage collection or an explicit close() call would now allow the finally clause to execute."

Since close method in generator is equivalent to throw a GeneratorExit and catched by yours exception, then logging.error('GeneratorExit') expression is executed.

"RunTimeError" is raised because the generator is yield next value n(9), it's state in python documentation https://docs.python.org/3.6/reference/expressions.html#generator-iterator-methods :

"Raises a GeneratorExit at the point where the generator function was paused. If the generator function then exits gracefully, is already closed, or raises GeneratorExit (by not catching the exception), close returns to its caller. If the generator yields a value, a RuntimeError is raised. If the generator raises any other exception, it is propagated to the caller. close() does nothing if the generator has already exited due to an exception or normal exit"

May be the code should like this :

#pygen.py

import sys
import logging
logging.basicConfig(level=logging.DEBUG, format='%(asctime)s \
        %(levelname)s - %(message)s', datefmt='[%Y/%m/%d %H:%M:%S]')

def genwrapper(func):
    #makes gen wrapper
    #which automatically send(None) to generator
    def wrapper(n=None):
        f = func(n)
        f.send(None)
        return f
    return wrapper

@genwrapper
def countdown(n=None):
    logging.debug('Counting Down')
    while True:
        try:
            n = yield(n)
        except GeneratorExit as e:
            logging.error('GeneratorExit')
            raise e

if __name__ == '__main__':
    n = int(sys.argv[1])
    c = countdown() #avoid function call in loop block (avoid new reference to c)
     while n > 0:
         a = c.send(n)
         logging.debug('Value: %d', a)
         n -= 1

then in yours terminal :

guest@xxxxpc:~$ python pygen.py 5

will result :

[2018/12/13 16:50:45]         DEBUG - Counting Down
[2018/12/13 16:50:45]         DEBUG - Value: 5
[2018/12/13 16:50:45]         DEBUG - Value: 4
[2018/12/13 16:50:45]         DEBUG - Value: 3
[2018/12/13 16:50:45]         DEBUG - Value: 2
[2018/12/13 16:50:45]         DEBUG - Value: 1
[2018/12/13 16:50:45]         ERROR - GeneratorExit

Sorry for my bad english or my suggestion if not clear enough, thanks

like image 24
Wira Bhakti Avatar answered Oct 18 '22 07:10

Wira Bhakti