How often should custom exceptions be defined in python?

Question

In trying to eliminate potential race condition in a python module I wrote to monitor some specialized workflows, I learned about python's "easier to ask forgiveness than permission" (EAFP) coding style, and I'm now raising lots of custom exceptions with try/except blocks where I used to use if/thens.

I'm new to python and This EAFP style makes sense logically and seems make my code more robust, but something about this feels way overboard. Is is bad practice to define one or more exceptions per method?

These custom exceptions tend to be useful only to a single method and, while it feels like a functionally correct solution, it seems like a lot of code to maintain.

Here a sample method for example:

class UploadTimeoutFileMissing(Exception):
    def __init__(self, value):
        self.parameter = value
    def __str__(self):
        return repr(self.parameter)

class UploadTimeoutTooSlow(Exception):
    def __init__(self, value):
        self.parameter = value
    def __str__(self):
        return repr(self.parameter)

def check_upload(file, timeout_seconds, max_age_seconds, min_age_seconds):

    timeout = time.time() + timeout_seconds

    ## Check until file found or timeout
    while (time.time() < timeout):

        time.sleep(5)
        try:
            filetime = os.path.getmtime(file)
            filesize = os.path.getsize(file)
        except OSError:
            print "File not found %s" % file
            continue

        fileage = time.time() - filetime

        ## Make sure file isn't pre-existing
        if fileage > max_age_seconds:
            print "File too old %s" % file
            continue

        ## Make sure file isn't still uploading
        elif fileage <= min_age_seconds:
            print "File too new %s" % file
            continue

        return(filetime, filesize)

    ## Timeout
    try:
        filetime
        filesize
        raise UploadTimeoutTooSlow("File still uploading")

    except NameError:
        raise UploadTimeoutFileMissing("File not sent")

Answer 1

define one or more exceptions per method

If you mean that the exception is actually defined per method as in "within the method body", then yes. That is bad practice. This is true also if you define two exceptions that would relate to the same error but you create two because two different methods raise them.

If you ask whether it is bad practice to raise more than one exception per method, then no, that is good practice. And if the errors are not of the same category, it's perfectly ok to define several exceptions per module.

In general, for larger modules you will define more than one exception. If you would work on some arithmetic library and you would define a ZeroDivisionError and an OverflowError (if they weren't already defined in python, because you can of course re-use those) that would be perfectly fine.

Answer 2

Is is bad practice to define one or more exceptions per method?

Yes.

One per module is more typical. It depends, of course, on the detailed semantics. The question boils down to this: "What will you really try to catch?"

If you're never going to use except ThisVeryDetailedException: in your code, then your very detailed exception isn't very helpful.

If you can do this: except Error as e: if e.some_special_case for the very few times it matters, then you can easily simplify to one exception per module and handle your special cases as attributes of the exception rather than different types of exceptions.

The common suggestions (one per module, named Error) means that your code will often look like this.

try: 
    something
except some_module.Error as e:
    carry on

This gives you a nice naming convention: module.Error. This covers numerous sins.

---

On an unrelated note, if you think you've got "potential race condition" you should probably redesign things correctly or stop trying to use threads or switch to multiprocessing. If you use multiprocessing, you'll find that it's very easy to avoid race conditions.