Exception handling over multiple calling levels

Question

How do you best handle multiple levels of methods in a call hierarchy that raise exceptions, so that if it is a fatal error the program will exit (after displaying an error dialog)?

I'm basically coming from Java. There I would simply declare any methods as throws Exception, re-throw it and catch it somewhere at the top level.

However, Python is different. My Python code basically looks like the below.

EDIT: added much simpler code...

Main entry function (plugin.py):

def main(catalog):

    print "Executing main(catalog)... "
    # instantiate generator
    gen = JpaAnnotatedClassGenerator(options)

    # run generator
    try:
        gen.generate_bar()  # doesn't bubble up
    except ValueError as error:
        Utilities.show_error("Error", error.message, "OK", "", "")
        return

    ... usually do the real work here if no error

JpaAnnotatedClassGenerator class (engine.py):

class JpaAnnotatedClassGenerator:

    def generate_bar(self):
        self.generate_value_error()

    def generate_value_error(self):
        raise ValueError("generate_value_error() raised an error!")

I'd like to return to the caller with an exception that is to be thrown back to that ones call until it reaches the outermost try-except to display an error dialog with the exception's message.

QUESTION: How is this best done in Python? Do I really have to repeat try-except for every method being called?

BTW: I am using Python 2.6.x and I cannot upgrade due to being bound to MySQL Workbench that provides the interpreter (Python 3 is on their upgrade list).

Answer 1

If you don't catch an exception, it bubbles up the call stack until someone does. If no one catches it, the runtime will get it and die with the exception error message and a full traceback. IOW, you don't have to explicitely catch and reraise your exception everywhere - which would actually defeat the whole point of having exceptions. Actually, despite being primarily used for errors / unexpected conditions, exceptions are first and foremost a control flow tool allowing to break out of the normal execution flow and pass control (and some informations) to any arbitrary place up in the call stack.

From this POV your code seems mostlt correct (caveat: I didn't bother reading the whole thing, just had a quick look), except (no pun indented) for a couple points:

First, you should define your own specific exception class(es) instead of using the builtin ValueError (you can inherit from it if it makes sense to you) so you're sure you only catch the exact exceptions you expect (quite a few layers "under" your own code could raise a ValueError that you didn't expect).

Then, you may (or not, depending on how your code is used) also want to add a catch-all top-level handler in your main() function so you can properly log (using the logger module) all errors and eventually free resources, do some cleanup etc before your process dies.

As a side note, you may also want to learn and use proper string formatting, and - if perfs are an issue at least -, avoid duplicate constant calls like this:

elif AnnotationUtil.is_embeddable_table(table) and AnnotationUtil.is_secondary_table(table):
    # ...
elif AnnotationUtil.is_embeddable_table(table):
    # ...
elif AnnotationUtil.is_secondary_table(table):
    # ...

Given Python's very dynamic nature, neither the compiler nor runtime can safely optimize those repeated calls (the method could have been dynamically redefined between calls), so you have to do it yourself.

EDIT:

When trying to catch the error in the main() function, exceptions DON'T bubble up, but when I use this pattern one level deeper, bubbling-up seems to work.

You can easily check that it works correctly with a simple MCVE:

def deeply_nested():
    raise ValueError("foo")

def nested():
    return deeply_nested()

def firstline():
    return nested()

def main():
    try:
        firstline()
    except ValueError as e:
        print("got {}".format(e))
    else:
        print("you will not see me")

if __name__ == "__main__":
    main()

It appears the software that supplies the Python env is somehow treating the main plugin file in a wrong way. Looks I will have to check the MySQL Workbench guys

Uhu... Even embedded, the mechanism expection should still work as expected - at least for the part of the call stack that depends on your main function (can't tell what happens upper in the call stack). But given how MySQL treats errors (what about having your data silently truncated ?), I wouldn't be specially suprised if they hacked the runtime to silently pass any error in plugins code xD

Answer 2

It is fine for errors to bubble up

Python's exceptions are unchecked, meaning you have no obligation to declare or handle them. Even if you know that something may raise, only catch the error if you intend to do something with it. It is fine to have exception-transparent layers, which gracefully abort as an exception bubbles through them:

def logged_get(map: dict, key: str):
    result = map[key]  # this may raise, but there is no state to corrupt
    # the following is not meaningful if an exception occurred
    # it is fine for it to be skipped by the exception bubbling up
    print(map, '[%s]' % key, '=>', result)
    return result

In this case, logged_get will simply forward any KeyError (and others) that are raised by the lookup. If an outer caller knows how to handle the error, it can do so.

So, just call self.create_collection_embeddable_class_stub the way you do.

It is fine for errors to kill the application

Even if nothing handles an error, the interpreter does. You get a stack trace, showing what went wrong and where. Fatal errors of the kind "only happens if there is a bug" can "safely" bubble up to show what went wrong.

In fact, exiting the interpreter and assertions use this mechanism as well.

>>> assert 2 < 1, "This should never happen"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
AssertionError: This should never happen

For many services, you can use this even in deployment - for example, systemd would log that for a Linux system service. Only try to suppress errors for the outside if security is a concern, or if users cannot handle the error.

It is fine to use precise errors

Since exceptions are unchecked, you can use arbitrary many without overstraining your API. This allows to use custom errors that signal different levels of problems:

class DBProblem(Exception):
    """Something is wrong about our DB..."""

class DBEntryInconsistent(DBProblem):
    """A single entry is broken"""

class DBInconsistent(DBProblem):
    """The entire DB is foobar!"""

It is generally a good idea not to re-use builtin errors, unless your use-case actually matches their meaning. This allows to handle errors precisely if needed:

try:
    gen.generate_classes(catalog)
except DBEntryInconsistent:
    logger.error("aborting due to corrupted entry")
    sys.exit(1)
except DBInconsistent as err:
    logger.error("aborting due to corrupted DB")
    Utility.inform_db_support(err)
    sys.exit(1)
# do not handle ValueError, KeyError, MemoryError, ...
# they will show up as a stack trace