Tracing code execution in embedded Python interpreter

Question

I'd like to create an application with embedded python interpreter and basic debugging capabilities. Now I'm searching the API for functions which I could use to run code step-by-step and get the number of the current line of code which is being (or is about to be) executed.

Official Python docs seem a little underdone for me when comes it to tracing and profiling. There is, for example, no information about the meaning of the return value of Py_tracefunc.

So far I've assembled the following:

#include <Python.h>

static int lineCounter = 0;

int trace(PyObject *obj, PyFrameObject *frame, int what, PyObject *arg)
{
    if(what == PyTrace_LINE)
    {
        lineCounter += 1;
        printf("line %d\n", lineCounter);
    }
    return 0;
}

int main(int argc, char *argv[])
{
    wchar_t *program = Py_DecodeLocale(argv[0], NULL);
    if (program == NULL) {
        fprintf(stderr, "Fatal error: cannot decode argv[0]\n");
        exit(1);
    }
    Py_SetProgramName(program);  /* optional but recommended */
    Py_Initialize();
    PyEval_SetTrace(trace, NULL);
    char *code = "def adder(a, b):\n"
                 " return a + b\n"
                 "x = 3\n"
                 "y = 4\n"
                 "print(adder(x, y))\n";
    PyRun_SimpleString(code);
    Py_Finalize();
    PyMem_RawFree(program);
    return 0;
}

However, the compiler outputs the following error:

hello.c:5:26: error: unknown type name ‘PyFrameObject’
 int trace(PyObject *obj, PyFrameObject *frame, int what, PyObject *arg)
                          ^

I'm operating on ManjaroLinux and using the following to compile the above:

gcc -o hello hello.c -I/usr/include/python3.5m  -Wno-unused-result -Wsign-compare -Wunreachable-code -march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong --param=ssp-buffer-size=4 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -L/usr/lib -lpython3.5m -lpthread -ldl  -lutil -lm  -Xlinker -export-dynamic

I've found that I can replace PyFrameObject with struct _frame and then program compiles but everyone knows it's a dirty hack, not a solution.

The executable outputs the following:

line 1
line 2
line 3
line 4
line 5
7

But I'd like the traces to follow the execution flow of the script (that is: start from line 3, then 4, 5 and then, due to the function call, 2).

I could not find anything about step-by-step execution.

Could you recommend some other sources about Python C API with more information and some introduction to the topic?

I awarded the answer with bounty since it would expire anyway. However, I'm still looking and would be grateful for answers for other questions from above.

Answer 1

hello.c:5:26: error: unknown type name ‘PyFrameObject’

This error means that PyFrameObject has not been declared. I did a Google search which showed me frameobject.h in the Python source tree is where that structure is declared.

I expect that you can add the line

#include <frameobject.h>

to resolve this.