"Uncatching" an exception in python

Question

How should I "rethrow" an exception, that is, suppose:

• I try something in my code, and unfortunately it fails.
• I try some "clever" workaround, which happens to also fail this time

If I throw the exception from the (failing) workaround, it's going to be pretty darn confusing for the user, so I think it may be best to rethrow the original exception (?), with the descriptive traceback it comes with (about the actual problem)...

Note: the motivating example for this is when calling np.log(np.array(['1'], dtype=object)), where it tries a witty workaround and gives an AttributeError (it's "really" a TypeError).

One way I can think of is just to re-call the offending function, but this seems doged (for one thing theoretically the original function may exert some different behaviour the second time it's called):
Okay this is one awful example, but here goes...

def f():
    raise Exception("sparrow")

def g():
    raise Exception("coconut")

def a():
    f()

Suppose I did this:

try:
    a()
except:
    # attempt witty workaround
    g()
---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
<ipython-input-4-c76b7509b315> in <module>()
      3 except:
      4     # attempt witty workaround
----> 5     g()
      6

<ipython-input-2-e641f2f9a7dc> in g()
      4
      5 def g():
----> 6     raise Exception("coconut")
      7
      8

Exception: coconut

Well, the problem doesn't really lie with the coconut at all, but the sparrow:

try:
    a()
except:
    # attempt witty workaround
    try:
        g()
    except:
        # workaround failed, I want to rethrow the exception from calling a()
        a() # ideally don't want to call a() again
---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
<ipython-input-4-e641f2f9a7dc> in <module>()
     19     except:
     20         # workaround failed, I want to rethrow the exception from calling a()
---> 21         a()  # ideally don't want to call a() again

<ipython-input-3-e641f2f9a7dc> in a()
      8
      9 def a():
---> 10     f()
     11
     12

<ipython-input-1-e641f2f9a7dc> in f()
      1 def f():
----> 2     raise Exception("sparrow")
      3
      4
      5 def g():

Exception: sparrow

Is there a standard way to deal with this, or am I thinking about it completely wrong?

Answer 1

If you want to make it appear to the end user that you never called g(), then you need to store the traceback from the first error, call the second function and then throw the original with the original traceback. (otherwise, in Python2, bare raise re-raises the second exception rather than the first). The problem is that there is no 2/3 compatible way to raise with traceback, so you have to wrap the Python 2 version in an exec statement (since it's a SyntaxError in Python 3).

Here's a function that lets you do that (I added this to the pandas codebase recently):

import sys
if sys.version_info[0] >= 3:
    def raise_with_traceback(exc, traceback=Ellipsis):
        if traceback == Ellipsis:
            _, _, traceback = sys.exc_info()
        raise exc.with_traceback(traceback)
else:
    # this version of raise is a syntax error in Python 3
    exec("""
def raise_with_traceback(exc, traceback=Ellipsis):
    if traceback == Ellipsis:
        _, _, traceback = sys.exc_info()
    raise exc, None, traceback
""")

raise_with_traceback.__doc__ = (
"""Raise exception with existing traceback.
If traceback is not passed, uses sys.exc_info() to get traceback."""
)

And then you can use it like this (I also changed the Exception types for clarity).

def f():
    raise TypeError("sparrow")

def g():
    raise ValueError("coconut")

def a():
    f()

try:
    a()
except TypeError as e:
    import sys
    # save the traceback from the original exception
    _, _, tb = sys.exc_info()
    try:
        # attempt witty workaround
        g()
    except:
        raise_with_traceback(e, tb)

And in Python 2, you only see a() and f():

Traceback (most recent call last):
  File "test.py", line 40, in <module>
    raise_with_traceback(e, tb)
  File "test.py", line 31, in <module>
    a()
  File "test.py", line 28, in a
    f()
  File "test.py", line 22, in f
    raise TypeError("sparrow")
TypeError: sparrow

But in Python 3, it still notes there was an additional exception too, because you are raising within its except clause [which flips the order of the errors and makes it much more confusing for the user]:

Traceback (most recent call last):
  File "test.py", line 38, in <module>
    g()
  File "test.py", line 25, in g
    raise ValueError("coconut")
ValueError: coconut

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "test.py", line 40, in <module>
    raise_with_traceback(e, tb)
  File "test.py", line 6, in raise_with_traceback
    raise exc.with_traceback(traceback)
  File "test.py", line 31, in <module>
    a()
  File "test.py", line 28, in a
    f()
  File "test.py", line 22, in f
    raise TypeError("sparrow")
TypeError: sparrow

If you absolutely want it to look like the g() Exception never happened in both Python 2 and Python 3, you need to check that you are out of the except clause first:

try:
    a()
except TypeError as e:
    import sys
    # save the traceback from the original exception
    _, _, tb = sys.exc_info()
    handled = False
    try:
        # attempt witty workaround
        g()
        handled = True
    except:
        pass
    if not handled:
        raise_with_traceback(e, tb)

Which gets you the following traceback in Python 2:

Traceback (most recent call last):
  File "test.py", line 56, in <module>
    raise_with_traceback(e, tb)
  File "test.py", line 43, in <module>
    a()
  File "test.py", line 28, in a
    f()
  File "test.py", line 22, in f
    raise TypeError("sparrow")
TypeError: sparrow

And this traceback in Python 3:

Traceback (most recent call last):
  File "test.py", line 56, in <module>
    raise_with_traceback(e, tb)
  File "test.py", line 6, in raise_with_traceback
    raise exc.with_traceback(traceback)
  File "test.py", line 43, in <module>
    a()
  File "test.py", line 28, in a
    f()
  File "test.py", line 22, in f
    raise TypeError("sparrow")
TypeError: sparrow

It does add an additional non-useful line of traceback that shows the raise exc.with_traceback(traceback) to the user, but it is relatively clean.

Answer 2

Here is something totally nutty that I wasn't sure would work, but it works in both python 2 and 3. (It does however, require the exception to be encapsulated into a function...)

def f():
    print ("Fail!")
    raise Exception("sparrow")
def g():
    print ("Workaround fail.")
    raise Exception("coconut")
def a():
    f()

def tryhard():
    ok = False
    try:
        a()
        ok = True
    finally:
        if not ok:
            try:
                g()
                return # "cancels" sparrow Exception by returning from finally
            except:
                pass

>>> tryhard()
Fail!
Workaround fail.
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 4, in tryhard
  File "<stdin>", line 2, in a
  File "<stdin>", line 3, in f
Exception: sparrow

Which is the correct exception and the right stack trace, and with no hackery.

>>> def g(): print "Worked around." # workaround is successful in this case

>>> tryhard()
Fail!
Worked around.

>>> def f(): print "Success!" # normal method works

>>> tryhard()
Success!