Getting original line number for exception in concurrent.futures

Question

Example of using concurrent.futures (backport for 2.7):

import concurrent.futures  # line 01
def f(x):  # line 02
    return x * x  # line 03
data = [1, 2, 3, None, 5]  # line 04
with concurrent.futures.ThreadPoolExecutor(len(data)) as executor:  # line 05
    futures = [executor.submit(f, n) for n in data]  # line 06
    for future in futures:  # line 07
        print(future.result())  # line 08

Output:

1
4
9
Traceback (most recent call last):
  File "C:\test.py", line 8, in <module>
    print future.result()  # line 08
  File "C:\dev\Python27\lib\site-packages\futures-2.1.4-py2.7.egg\concurrent\futures\_base.py", line 397, in result
    return self.__get_result()
  File "C:\dev\Python27\lib\site-packages\futures-2.1.4-py2.7.egg\concurrent\futures\_base.py", line 356, in __get_result
    raise self._exception
TypeError: unsupported operand type(s) for *: 'NoneType' and 'NoneType'

String "...\_base.py", line 356, in __get_result" is not endpoint I expected to see. Is it possible to get real line where exception was thrown? Something like:

  File "C:\test.py", line 3, in f
    return x * x  # line 03

Python3 seems to show correct line number in this case. Why can't python2.7? And is there any workaround?

Answer 1

I was in your same situation and I really needed to have the traceback of the raised exceptions. I was able to develop this workaround which consists in using the following subclass of the ThreadPoolExecutor.

import sys
import traceback
from concurrent.futures import ThreadPoolExecutor

class ThreadPoolExecutorStackTraced(ThreadPoolExecutor):

    def submit(self, fn, *args, **kwargs):
        """Submits the wrapped function instead of `fn`"""

        return super(ThreadPoolExecutorStackTraced, self).submit(
            self._function_wrapper, fn, *args, **kwargs)

    def _function_wrapper(self, fn, *args, **kwargs):
        """Wraps `fn` in order to preserve the traceback of any kind of
        raised exception

        """
        try:
            return fn(*args, **kwargs)
        except Exception:
            raise sys.exc_info()[0](traceback.format_exc())  # Creates an
                                                             # exception of the
                                                             # same type with the
                                                             # traceback as
                                                             # message

If you use this subclass and run the following snippet:

def f(x):
    return x * x

data = [1, 2, 3, None, 5]
with ThreadPoolExecutorStackTraced(max_workers=len(data)) as executor:
    futures = [executor.submit(f, n) for n in data]
    for future in futures:
        try:
            print future.result()
        except TypeError as e:
            print e

the output will be something like:

1
4
9
Traceback (most recent call last):
  File "future_traceback.py", line 17, in _function_wrapper
    return fn(*args, **kwargs)
  File "future_traceback.py", line 24, in f
    return x * x
TypeError: unsupported operand type(s) for *: 'NoneType' and 'NoneType'

25

The problem is in the usage of sys.exc_info() by the futures library. From the documentation:

This function returns a tuple of three values that give information about the exception that is currently being handled. [...] If no exception is being handled anywhere on the stack, a tuple containing three None values is returned. Otherwise, the values returned are (type, value, traceback). Their meaning is: type gets the exception type of the exception being handled (a class object); value gets the exception parameter (its associated value or the second argument to raise, which is always a class instance if the exception type is a class object); traceback gets a traceback object which encapsulates the call stack at the point where the exception originally occurred.

Now, if you look at the source code of futures you can see by yourself why the traceback is lost: when an exception raises and it is to be set to the Future object only sys.exc_info()[1] is passed. See:

https://code.google.com/p/pythonfutures/source/browse/concurrent/futures/thread.py (L:63) https://code.google.com/p/pythonfutures/source/browse/concurrent/futures/_base.py (L:356)

So, to avoid losing the traceback, you have to save it somewhere. My workaround is to wrap the function to submit into a wrapper whose only task is to catch every kind of exception and to raise an exception of the same type whose message is the traceback. By doing this, when an exception is raised it is captured and reraised by the wrapper, then when sys.exc_info()[1] is assigned to the exception of the Future object, the traceback is not lost.

Answer 2

I think the original exception traceback gets lost in the ThreadPoolExecutor code. It stores the exception and then re-raises it later. Here is one solution. You can use the traceback module to store the original exception message and traceback from your function f into a string. Then raise an exception with this error message, which now contains the line number etc of f. The code that runs f can be wrapped in a try...except block, which catches the exception raised from ThreadPoolExecutor, and prints the message, which contains the original traceback.

The code below works for me. I think this solution is a little hacky, and would prefer to be able to recover the original traceback, but I'm not sure if that is possible.

import concurrent.futures
import sys,traceback


def f(x):
    try:
        return x * x
    except Exception, e:
        tracebackString = traceback.format_exc(e)
        raise StandardError, "\n\nError occurred. Original traceback is\n%s\n" %(tracebackString)



data = [1, 2, 3, None, 5]  # line 10

with concurrent.futures.ThreadPoolExecutor(len(data)) as executor:  # line 12
    try:
        futures = [executor.submit(f, n) for n in data]  # line 13
        for future in futures:  # line 14
           print(future.result())  # line 15
    except StandardError, e:
        print "\n"
        print e.message
        print "\n"

This gives the following output in python2.7:

1
4
9




Error occurred. Original traceback is
Traceback (most recent call last):
File "thread.py", line 8, in f
   return x * x
TypeError: unsupported operand type(s) for *: 'NoneType' and 'NoneType'

The reason your original code gives the right location when run in Python 3 and not 2.7 is that in Python 3 exceptions carry the traceback as an attribute, and when re-raising an exception, the traceback is extended rather than replaced. The example below illustrates this:

def A():
    raise BaseException("Fish")

def B():
    try:
        A()
    except BaseException as e:
        raise e

B()

I ran this in python 2.7 and python 3.1. In 2.7 the output is as follows:

Traceback (most recent call last):
  File "exceptions.py", line 11, in <module>
    B()
  File "exceptions.py", line 9, in B
    raise e
BaseException: Fish

i.e. the fact that the exception was originally thrown from A is not recorded in the eventual output. When I run with python 3.1 I get this:

Traceback (most recent call last):
  File "exceptions.py", line 11, in <module>
    B()
  File "exceptions.py", line 9, in B
    raise e
  File "exceptions.py", line 7, in B
    A()
  File "exceptions.py", line 3, in A
    raise BaseException("Fish")
BaseException: Fish

which is better. If I replace raise e with just raise in the except block in B, then python2.7 gives the complete traceback. My guess is that when back-porting this module for python2.7 the differences in exception propagating were overlooked.

Answer 3

Taking inspiration from the first answer, here it is as a decorator:

import functools
import traceback


def reraise_with_stack(func):

    @functools.wraps(func)
    def wrapped(*args, **kwargs):
        try:
            return func(*args, **kwargs)
        except Exception as e:
            traceback_str = traceback.format_exc(e)
            raise StandardError("Error occurred. Original traceback "
                                "is\n%s\n" % traceback_str)

    return wrapped

Just apply the decorator on the executed function:

@reraise_with_stack
def f():
    pass