Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Exception TypeError warning sometimes shown, sometimes not when using throw method of generator

There is this code:

class MyException(Exception):
  pass

def gen():
  for i in range(3):
    try:
      yield i
    except MyException:
      print("MyException!")


a = gen()
next(a) 
a.throw(MyException)

Running this code:

$ python3.3 main.py
MyException!
$ python3.3 main.py
MyException!
Exception TypeError: TypeError('catching classes that do not inherit from BaseException is not allowed',) in <generator object gen at 0xb712efa4> ignored
$ python3.3 main.py
MyException!
$ python3.3 main.py
MyException!
$ python3.3 main.py
MyException!
Exception TypeError: TypeError('catching classes that do not inherit from BaseException is not allowed',) in <generator object gen at 0xb714afa4> ignored

The thing which I don't understand is why sometimes there is printed this Exception TypeError warning. Is there something wrong with custom exception?

like image 923
scdmb Avatar asked Aug 10 '13 16:08

scdmb


People also ask

What is the exception class in Python?

In Python, all exceptions must be instances of a class that derives from BaseException . In a try statement with an except clause that mentions a particular class, that clause also handles any exception classes derived from that class (but not exception classes from which it is derived).

What is except exception as e?

except: accepts all exceptions, whereas. except Exception as e: only accepts exceptions that you're meant to catch. Here's an example of one that you're not meant to catch: >>> try: ...

What is TypeError in Python?

TypeError is one among the several standard Python exceptions. TypeError is raised whenever an operation is performed on an incorrect/unsupported object type. For example, using the + (addition) operator on a string and an integer value will raise TypeError.


3 Answers

You are seeing a __del__ hook misbehaving somewhere.

The TypeError is being thrown while shutting down, as the Python interpreter is exiting everything is deleted and any exceptions thrown in a __del__ deconstructor hook are being ignored (but are printed).

On exit, Python clears everything in the namespace by rebinding everything to None, but the order in which this happens is not set. The still running generator is closed (a.close() is called) when deleted, which triggers a GeneratorExit exception in the generator, which Python tests against your except MyException: line. If, however, MyException has already been cleared up and Python sees except None: the TypeError is thrown and you see that message printed.

You can trigger the error without exiting Python by adding:

MyException = None
del a

If you use list(a) and consume the rest of the generator, or explicitly close the generator with a.close() before Python exits and deletes MyException, the error message goes away.

Another work-around would be to handle GeneratorExit first:

def gen():
  for i in range(3):
    try:
      yield i
    except GeneratorExit:
      return
    except MyException:
      print("MyException!")

and Python will not evaluate the next except handler.

The error cannot be reproduced with Python 3.2 or earlier, so it looks like hash randomization (introduced in Python 3.3) randomizes the order objects are cleared; this certainly explains why you see the error only on some of your runs, but not on earlier Python runs where the hash order is fixed.

Note that the interaction of .__del__() hooks and other global objects in Python is documented with a big red warning in the .__del__() documentation:

Warning: Due to the precarious circumstances under which __del__() methods are invoked, exceptions that occur during their execution are ignored, and a warning is printed to sys.stderr instead. Also, when __del__() is invoked in response to a module being deleted (e.g., when execution of the program is done), other globals referenced by the __del__() method may already have been deleted or in the process of being torn down (e.g. the import machinery shutting down). For this reason, __del__() methods should do the absolute minimum needed to maintain external invariants. Starting with version 1.5, Python guarantees that globals whose name begins with a single underscore are deleted from their module before other globals are deleted; if no other references to such globals exist, this may help in assuring that imported modules are still available at the time when the __del__() method is called.

like image 173
Martijn Pieters Avatar answered Nov 04 '22 03:11

Martijn Pieters


I was having this same error in Python 3.3 on Windows, with the difference that I was defining the exception in its own file. These were my code files:

$ cat FooError.py 
class FooError(Exception):
    pass

$ cat application.py
import FooError
try:
    raise FooError('Foo not bar!')
Except FooError as e:
    print(e)

This was the exception that I was getting:

TypeError: catching classes that do not inherit from BaseException is not allowed.

Changing import FooError to from FooError import * resolved the issue. Here is the final code, for clarity's sake:

$ cat FooError.py 
class FooError(Exception):
    pass

$ cat application.py
from FooError import *
try:
    raise FooError('Foo not bar!')
Except FooError as e:
    print(e)
like image 25
dotancohen Avatar answered Nov 04 '22 03:11

dotancohen


I had the same problem - but I was missing the import to the exception class. So the interpreter didn't resolve the class on the except clause.

So just add the import and hopefully everything will work.

like image 23
Guy L Avatar answered Nov 04 '22 04:11

Guy L