Python generator, non-swallowing exception in 'coroutine'

Question

I recently came across some surprising behaviour in Python generators:

class YieldOne:
  def __iter__(self):
    try:
      yield 1
    except:
      print '*Excepted Successfully*'
      # raise

for i in YieldOne():
  raise Exception('test exception')

Which gives the output:

*Excepted Successfully*
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
Exception: test exception

I was (pleasantly) surprised that *Excepted Successfully* got printed, as this was what I wanted, but also surprised that the Exception still got propagated up to the top level. I was expecting to have to use the (commented in this example) raise keyword to get the observed behaviour.

Can anyone explain why this functionality works as it does, and why the except in the generator doesn't swallow the exception?

Is this the only instance in Python where an except doesn't swallow an exception?

Answer 1

Your code does not do what you think it does. You cannot raise Exceptions in a coroutine like this. What you do instead is catching the GeneratorExit exception. See what happens when you use a different Exception:

class YieldOne:
  def __iter__(self):
    try:
      yield 1
    except RuntimeError:
        print "you won't see this"
    except GeneratorExit:
      print 'this is what you saw before'
      # raise

for i in YieldOne():
  raise RuntimeError

As this still gets upvotes, here is how you raise an Exception in a generator:

class YieldOne:
  def __iter__(self):
    try:
      yield 1
    except Exception as e:
      print "Got a", repr(e)
      yield 2
      # raise

gen = iter(YieldOne())

for row in gen:
    print row # we are at `yield 1`
    print gen.throw(Exception) # throw there and go to `yield 2` 

See docs for generator.throw.

Answer 2

EDIT: What THC4k said.

If you really want to raise an arbitrary exception inside a generator, use the throw method:

>>> def Gen():
...     try:
...             yield 1
...     except Exception:
...             print "Excepted."
...
>>> foo = Gen()
>>> next(foo)
1
>>> foo.throw(Exception())
Excepted.
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration

You'll notice that you get a StopIteration at the top level. These are raised by generators which have run out of elements; they are usually swallowed by the for loop but in this case we made the generator raise an exception so the loop doesn't notice them.