Why does handling multiple exceptions require a tuple, but not a list?

Question

Consider the following example:

def main_list(error_type):

    try:
        if error_type == 'runtime':
            raise RuntimeError("list error")
        if error_type == 'valueerror':
            raise ValueError("list error")

    except [RuntimeError, ValueError] as e:
        print str(e)

def main_tuple(error_type):

    try:
        if error_type == 'runtime':
            raise RuntimeError("tuple error")
        if error_type == 'valueerror':
            raise ValueError("tuple error")

    except (RuntimeError, ValueError) as e:
        print str(e)


main_tuple('runtime')
main_tuple('valueerror')

main_list('runtime')
main_list('valueerror')

The tuple is the correct way to handle multiple exception types. Using a list for the multiple exception types causes neither to be handled.

I am wondering why Python syntax requires a tuple for multiple exception types. The docs say that it uses a tuple, so perhaps it is just "never was implemented using a list instead of a tuple."

It seems reasonable to me that a list could also be used in this situation, conceptually at least.

Is there any reason why Python uses a tuple instead of a list for this situation?

Answer 1

Why does handling multiple exceptions require a tuple and not a list?

The error handling, written in C, uses type checking for the special case of a tuple, before other type-checking and exception handling, so that multiple types of exceptions may be caught.

At least one Python core developer advocates using exception handling for control flow. Adding lists as an additional type to check would work against this strategy.

It appears that expanding this to allow for sets or lists has not been specifically addressed by the core development team, though I will gladly reference it if it can be found. There has been a discussion on the Python mailing list that speculates quite a lot (another answer here quotes one response at length).

After performing the below analysis, and in the context of the mailing list discussion, I think the reasoning is obvious. I do not suggest proposing to add other containers.

Demonstration that lists fail versus tuples

exceptions = TypeError, RuntimeError
list_of_exceptions = list(exceptions)

Catching the tuple of exceptions does work:

try:
    raise TypeError('foo')
except exceptions as error:
    print(error)

outputs:

foo

But catching the list of exceptions does not work:

try:
    raise TypeError('foo')
except list_of_exceptions as error:
    print(error)

prints:

Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
TypeError: foo

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 3, in <module>
TypeError: catching classes that do not inherit from BaseException is not allowed

This demonstrates that we are doing type checking for the special case of a tuple. It would certainly make the code slower to add another type to check for, and the core developers have been saying that it's a good thing to use exception handling for control flow in Python for a while now.

Source Code analysis

An analysis of the source agrees with this above conclusion.

Grammar

This is not an issue for Python's grammar or parsing. It will accept any expression. Thus any expression that results in an Exception or tuple of Exceptions should be legal.

Disassembly

If we disassemble a function that does this in Python 3, we see that it looks to match an exception with a comparison operation.

def catch(exceptions):
    try:
        raise Exception
    except exceptions:
        pass

import dis
dis.dis(catch)

Which outputs:

      2           0 SETUP_EXCEPT            10 (to 13)

      3           3 LOAD_GLOBAL              0 (Exception)
                  6 RAISE_VARARGS            1
                  9 POP_BLOCK
                 10 JUMP_FORWARD            18 (to 31)

      4     >>   13 DUP_TOP
                 14 LOAD_FAST                0 (exceptions)
                 17 COMPARE_OP              10 (exception match)
    ...

This leads us inside the Python interpreter.

Internal control flow - CPython's implementation details

The CPython control flow first checks for if the value is a tuple. If so, it iterates through the tuple using tuple specific code - looking for the value to be a Exception:

case PyCmp_EXC_MATCH:
    if (PyTuple_Check(w)) {
        Py_ssize_t i, length;
        length = PyTuple_Size(w);
        for (i = 0; i < length; i += 1) {
            PyObject *exc = PyTuple_GET_ITEM(w, i);
            if (!PyExceptionClass_Check(exc)) {
                _PyErr_SetString(tstate, PyExc_TypeError,
                                 CANNOT_CATCH_MSG);
                return NULL;
            }
        }
    }
    else {
        if (!PyExceptionClass_Check(w)) {
            _PyErr_SetString(tstate, PyExc_TypeError,
                             CANNOT_CATCH_MSG);
            return NULL;
        }
    }
    res = PyErr_GivenExceptionMatches(v, w);
    break;

Adding another type would require more internal control flow, slowing down control flow inside of the Python interpreter.

Sizes of Python Containers

Tuples are lightweight arrays of pointers. So are lists, but they may be allocated extra space so that you may quickly add to them (up to the point they need to get larger). In Python 3.7.3 on Linux:

>>> from sys import getsizeof
>>> getsizeof((1,2,3))
72
>>> getsizeof([1,2,3])
88

Sets take up even more space because they are hash tables. They have both a hash of the object they contain as well as a pointer to that which they point to.

Conclusion

This is for the CPython core development team to debate and decide.

But my conclusion is that slowing down control flow in Python by checking for other types even at the C level would work against the strategy of using exception handling for control flow in Python modules.

After reasoning through the above, I would not propose that they add this.

Answer 2

@BrenBarn Thanks for your link to the discussion at https://mail.python.org/pipermail/python-list/2012-January/619107.html

I think the best and clearest response comes from Steven D'Aprano's reply at https://mail.python.org/pipermail/python-list/2012-January/619120.html

I copied the content below for easy read.

---

Steven's reply:

Simplicity.

If you also allow lists, then why not allow arbitrary sequences? What about iterators, do you allow them? That could be awkward, because iterators can only be run through once. Dictionaries are also iterable, so once you allow arbitrary iterables, you get dicts. The whole thing becomes a mess. Better to keep it simple and only allow a single canonical collection type, and in Python, that type is tuple, not list.

Tuples are that canonical collection type because they have a number of desirable properties:

• Tuples are small and memory efficient, using the smallest amount of memory needed to hold their items. Lists typically carry a block of spare memory, to make insertions fast.

• Consequently the Python virtual machine can create them rapidly and efficiently.

• Tuples are immutable, so you don't have to worry about passing one to a function and having the function modify it behind your back.

• Tuples are ordered, for the times where that matters.

• Since the typical use-case is to iterate over the items in fixed order, there's no need to pay the extra expense for a dict or set.

• Tuples are simple to write: in general you only need commas between items. Sometimes, to avoid ambiguity or change the precedence of calculation, you also need round brackets (parentheses for Americans). Except clauses are one of those times.

• Frozensets and sets are ruled out for historical reasons: they didn't exist until Python 2.3. Besides, which would you rather write?

  ("abc", "def") frozenset([abc", "def"])

• Sets and lists are ruled out because they are mutable, both require much more memory, and sets have a heavier computational burden.

The latter makes more sense semantically to me -- "catch all exception types in a list" as opposed to "catch this single thing composed of three exception types".

Then you are labouring under a misunderstanding. You're not catching a tuple, because tuples are never thrown. You're catching any of the exceptions that are contained in that tuple.

Both lists and tuples are single things in themselves. Both lists and tuples are containers:

A list is a single thing that contains other things.

A tuple is a single thing that contains other things.