File "<string>" traceback with line preview

Question

I'm making a couple of functions via exec, that could possibly error. However when Python errors out it doesn't display the line that the error occurred on.

For example using:

fn_str = '''\
def fn():
    raise Exception()
'''

globs = {}
exec(fn_str, globs)
fn = globs['fn']

fn()

Gives us the output:

Traceback (most recent call last):
  File "...", line 10, in <module>
    fn()
  File "<string>", line 2, in fn
Exception

---

If we however, don't use eval. Then we get the line that the program errored on:

def fn():
    raise Exception()

fn()

Traceback (most recent call last):
  File "...", line 4, in <module>
    fn()
  File "...", line 2, in fn
    raise Exception()
Exception

---

I've looked into using the __traceback__, however I couldn't find a way to add to the traceback under the 'File' line. And so the best I could get was this:

fn_str = '''\
def fn():
    try:
        raise Exception()
    except BaseException as e:
        tb = e.__traceback__
        if 1 <= tb.tb_lineno <= len(fn_lines):
            e.args = ((e.args[0] if e.args else '') + ' - ' + fn_lines[tb.tb_lineno - 1].strip(),)
        raise
'''

globs = {'fn_lines': fn_str.split('\n')}
exec(fn_str, globs)
fn = globs['fn']

fn()

Traceback (most recent call last):
  File "...", line 16, in <module>
    fn()
  File "<string>", line 3, in fn
Exception:  - raise Exception()

The biggest problem with this is if the eval calls other code, it becomes confusing where the - raise Exception() comes from.

---

Is there a way to make the eval code provide the line that it errored on?

Answer 1

The missing lines are a symptom of the fact that Python cannot find a file on disk named <string>, which is the filename built into your compiled snippet of code. (Though if you create a file with exactly that name, Python will print lines from it!)

Approach 1. You can catch exceptions yourself, whether at the top level of your application or elsewhere, and instead of letting the default builtin traceback routine fire you can call the Standard Library routine traceback.print_exc() which pulls lines from the Standard Library module linecache. Because the linecache cache is a simple public Python dictionary, you can pre-populate it with the source code it needs to print. See:

Why does the Python linecache affect the traceback module but not regular tracebacks?

The resulting code:

import linecache
import traceback

source = 'print("Hello, world" + 1)'
source_name = 'Example'
lines = source.splitlines(True)
linecache.cache[source_name] = len(source), None, lines, source_name
compiled = compile(source, source_name, 'exec')
try:
    eval(compiled)
except Exception:
    traceback.print_exc()

Approach 2. You can also avoid the indirection of populating a global cache by simply taking charge of printing the exception yourself: you can have Python return the traceback data as a list of tuples, step through them adding the missing lines, and then finally print them as usual.

Here is a fill_in_lines() function that fills out the traceback with the missing information, in a small program that prints the full traceback:

import sys
import traceback

def fill_in_lines(frames, source_name, source):
    lines = source.splitlines()
    for filename, line_number, function_name, text in frames:
        if filename == source_name:
            text = lines[line_number - 1]
        yield filename, line_number, function_name, text

source = 'print("Hello, world" + 1)'
source_name = 'Example'
compiled = compile(source, source_name, 'exec')
try:
    eval(compiled)
except Exception as e:
    _, _, tb = sys.exc_info()
    frames = traceback.extract_tb(tb)
    frames = fill_in_lines(frames, source_name, source)

    print('Traceback (most recent call last):')
    print(''.join(traceback.format_list(frames)), end='')
    print('{}: {}'.format(type(e).__name__, str(e)))

I am able to use the fancy name “Example” here because I set it using compile(). In your case you would instead want to pass the bare string '<string>' as the source_name argument.