Accessing a Python traceback from the C API

Question

I'm having some trouble figuring out the proper way to walk a Python traceback using the C API. I'm writing an application that embeds the Python interpreter. I want to be able to execute arbitrary Python code, and if it raises an exception, to translate it to my own application-specific C++ exception. For now, it is sufficient to extract just the file name and line number where the Python exception was raised. This is what I have so far:

PyObject* pyresult = PyObject_CallObject(someCallablePythonObject, someArgs);
if (!pyresult)
{
    PyObject* excType, *excValue, *excTraceback;
    PyErr_Fetch(&excType, &excValue, &excTraceback);
    PyErr_NormalizeException(&excType, &excValue, &excTraceback);

    PyTracebackObject* traceback = (PyTracebackObject*)traceback;
    // Advance to the last frame (python puts the most-recent call at the end)
    while (traceback->tb_next != NULL)
        traceback = traceback->tb_next;

    // At this point I have access to the line number via traceback->tb_lineno,
    // but where do I get the file name from?

    // ...       
}

Digging around in the Python source code, I see they access both the filename and module name of the current frame via the _frame structure, which looks like it is a privately-defined struct. My next idea was to programmatically load the Python 'traceback' module and call its functions with the C API. Is this sane? Is there a better way to access a Python traceback from C?

Answer 1

This is an old question but for future reference, you can get the current stack frame from the thread state object and then just walk the frames backward. A traceback object isn't necessary unless you want to preserve the state for the future.

For example:

PyThreadState *tstate = PyThreadState_GET(); if (NULL != tstate && NULL != tstate->frame) {     PyFrameObject *frame = tstate->frame;      printf("Python stack trace:\n");     while (NULL != frame) {         // int line = frame->f_lineno;         /*          frame->f_lineno will not always return the correct line number          you need to call PyCode_Addr2Line().         */         int line = PyCode_Addr2Line(frame->f_code, frame->f_lasti);         const char *filename = PyString_AsString(frame->f_code->co_filename);         const char *funcname = PyString_AsString(frame->f_code->co_name);         printf("    %s(%d): %s\n", filename, line, funcname);         frame = frame->f_back;     } } 

Answer 2

I prefer calling into python from C:

err = PyErr_Occurred();
if (err != NULL) {
    PyObject *ptype, *pvalue, *ptraceback;
    PyObject *pystr, *module_name, *pyth_module, *pyth_func;
    char *str;

    PyErr_Fetch(&ptype, &pvalue, &ptraceback);
    pystr = PyObject_Str(pvalue);
    str = PyString_AsString(pystr);
    error_description = strdup(str);

    /* See if we can get a full traceback */
    module_name = PyString_FromString("traceback");
    pyth_module = PyImport_Import(module_name);
    Py_DECREF(module_name);

    if (pyth_module == NULL) {
        full_backtrace = NULL;
        return;
    }

    pyth_func = PyObject_GetAttrString(pyth_module, "format_exception");
    if (pyth_func && PyCallable_Check(pyth_func)) {
        PyObject *pyth_val;

        pyth_val = PyObject_CallFunctionObjArgs(pyth_func, ptype, pvalue, ptraceback, NULL);

        pystr = PyObject_Str(pyth_val);
        str = PyString_AsString(pystr);
        full_backtrace = strdup(str);
        Py_DECREF(pyth_val);
    }
}

Answer 3

I've discovered that _frame is actually defined in the frameobject.h header included with Python. Armed with this plus looking at traceback.c in the Python C implementation, we have:

#include <Python.h>
#include <frameobject.h>

PyTracebackObject* traceback = get_the_traceback();

int line = traceback->tb_lineno;
const char* filename = PyString_AsString(traceback->tb_frame->f_code->co_filename);

But this still seems really dirty to me.

Answer 4

One principal I've found useful in writing C extensions is to use each language where it's best suited. So if you have a task to do that would be best implemented in Python, implement in Python, and if it would be best implemented in C, do it in C. Interpreting tracebacks is best done in Python for two reasons: first, because Python has the tools to do it, and second, because it isn't speed-critical.

I would write a Python function to extract the info you need from the traceback, then call it from C.

You could even go so far as to write a Python wrapper for your callable execution. Instead of invoking someCallablePythonObject, pass it as an argument to your Python function:

def invokeSomeCallablePythonObject(obj, args):
    try:
        result = obj(*args)
        ok = True
    except:
        # Do some mumbo-jumbo with the traceback, etc.
        result = myTraceBackMunger(...)
        ok = False
    return ok, result

Then in your C code, call this Python function to do the work. The key here is to decide pragmatically which side of the C-Python split to put your code.