Python Exceptions: EAFP and What is Really Exceptional?

Question

It's been said in a couple places (here and here) that Python's emphasis on "it's easier to ask for forgiveness than permission" (EAFP) should be tempered with the idea that exceptions should only be called in truly exceptional cases. Consider the following, in which we're popping and pushing on a priority queue until only one element is left:

import heapq
...
pq = a_list[:]
heapq.heapify(pq)
while True:
    min1 = heapq.heappop(pq)
    try:
        min2 = heapq.heappop(pq)
    except IndexError:
        break
    else
        heapq.heappush(pq, min1 + min2)
# do something with min1

The exception is only raised once in len(a_list) iterations of the loop, but it's not really exceptional, because we know its going to happen eventually. This setup saves us from checking whether a_list is empty a bunch of times, but (maybe) it's less readable than using explicit conditions.

What's the consensus on using exceptions for this kind of non-exceptional program logic?

Answer 1

exceptions should only be called in truly exceptional cases

Not in Python: for example, every for loop (unless it prematurely breaks or returns) terminates by an exception (StopIteration) being thrown and caught. So, an exception that happens once per loop is hardly strange to Python -- it's there more often than not!

The principle in question may be crucial in other languages, but that's definitely no reason to apply that principle to Python, where it's so contrary to the language's ethos.

In this case I like Jon's rewrite (which should be further simplified by removing the else branch) because it makes the code more compact -- a pragmatical reason, most definitely not the "tempering" of Python style with an alien principle.

Answer 2

Throwing exceptions is expensive in most low-level languages like C++. That influences a lot of the "common wisdom" about exceptions, and doesn't apply so much to languages that run in a VM, like Python. There's not such a major cost in Python for using an exception instead of a conditional.

(This is a case where the "common wisdom" becomes a matter of habit. People come to it from experience in one type of environment--low-level languages--and then apply it to new domains without evaluating whether it makes sense.)

Exceptions are still, in general, exceptional. That doesn't mean that they don't happen often; it means that they're the exception. They're the things that will tend to break from ordinary code flow, and which most of the time you don't want to have to handle one by one--which is the point of exception handlers. This part is the same in Python as in C++ and all other languages with exceptions.

However, that tends to define when exceptions are thrown. You're talking about when exceptions should be caught. Very simply, don't worry about it: exceptions aren't expensive, so don't go to lengths to try to prevent them from being thrown. A lot of Python code is designed around this.

I don't agree with Jon's suggestion to try to test for and avoid exceptions in advance. That's fine if it leads to clearer code, as in his example. However, in many cases it's just going to complicate things--it can effectively lead to duplicating checks and introducing bugs. For example,

import os, errno, stat

def read_file(fn):
    """
    Read a file and return its contents.  If the file doesn't exist or
    can't be read, return "".
    """
    try:
        return open(fn).read()
    except IOError, e:
        return ""

def read_file_2(fn):
    """
    Read a file and return its contents.  If the file doesn't exist or
    can't be read, return "".
    """
    if not os.access(fn, os.R_OK):
        return ""
    st = os.stat(fn)
    if stat.S_ISDIR(st.st_mode):
        return ""
    return open(fn).read()

print read_file("x")

Sure, we can test for and avoid the failure--but we've complicated things badly. We're trying to guess all the ways the file access might fail (and this doesn't catch all of them), we may have introduced race conditions, and we're doing a lot more I/O work. This is all done for us--just catch the exception.