Passing a C pointer around with the Python/C API

Question

I'm new to the Python/C API ... I'm trying to add new functionality to my C program, wherein I can embed python into it and simultaneously extend functionality so that the embedded interpreter can execute a script that will interact with an extending python module written as part of my C program. My C program doesn't have global variables. I would like to keep things this way; At the same time in order to expose C functionality to python, it appears the extending C function at least needs access to global variables to access state of the program. How do I get around this?

e.g. Here is how I plan on embedding where PYINTERFACE_Initialize is called from main

void PYINTERFACE_Initialize(State *ptr, FILE *scriptFile, const char* scriptFileName)
{
    Py_Initialize();
    PyObject *module = Py_InitModule("CInterface", CInterfaceMethods);
    if (PyRun_SimpleFileEx(scriptFile, scriptFileName, 1) != 0)
    {
        printf("PYINTERFACE script execution failed!\n");
    }
    **//ADD State *ptr TO module**      
}

Here is the extended function:

static PyObject*
CInterface_GetStateInfo(PyObject *self, PyObject *args)
{
    const char *printStr;
    double stateInfo;
    State *ptr;

    if(!PyArg_ParseTuple(args, "s", &printStr))
    {
        return NULL;
    }
    printf("Received %s\n", printStr);

    **//RETRIEVE State *ptr FROM self**      

    stateInfo = ptr->info;
    return Py_BuildValue("d", currentTime);
}

Is this the cleanest way to pass the State *ptr around? I certainly don't see the need to expose the internal state to python. I've thought about using capsules, but it doesn't seem to be the intention of capsules to support this sort of behaviour.

Thanks in advance! V

Answer 1

Capsules are basically python-opaque void pointers that you can pass around or associate with modules. They are "the way" to solve your problem.

Here's an example that uses an instance x that doesn't have to be static. First attach the pointer to your module something like this (error checking removed)...

// wrap the methods to be exposed to python in a module
// i.e. this is a list of method descriptions for the module
static PyMethodDef InitializeTurkeyMethods[] = {

    // this block describes one method.. turkey.do_something()
    {"do_something", 
     turkey_do_something, // fn pointer to wrap (defined below)
     METH_VARARGS, 
     "do something .. return an int."},

    {NULL, NULL, 0, NULL} // sentinel.
};


int init(X * x) { 

    // initialize embedded python scripting .. 
    // (this method a no-op on second or later calls).
    Py_Initialize();

    // initialize the turkey python module 
    PyObject * module = Py_InitModule("turkey", InitializeTurkeyMethods);

    // Create a capsule containing the x pointer 
    PyObject * c_api_object = PyCapsule_New((void *)x, "turkey._X_C_API", NULL);

    // and add it to the module
    PyModule_AddObject(module, "_X_C_API", c_api_object);
}

Then in the function you want to expose to python in order to get that X pointer back you do something like this (this actually has to go before you start referring to it in the code above):

static PyObject* turkey_do_something(PyObject *self, PyObject *args) {    

    if(!PyArg_ParseTuple(args, ":turkey_do_something"))
        return NULL;

    // get the x pointer back from the capsule
    X * x = (X*)PyCapsule_Import("turkey._X_C_API", 0);    

    // call some fn on x 
    return Py_BuildValue("i", x->some_fn_that_returns_an_int());
}

Here "turkey._X_C_API" is just a name for some added type checking - put some meaningful name in here for your app. Turkey is a demo module name that I made up just then.

Now assuming that, and depending on how, you've exported the turkey_do_something fn when calling Py_InitModule() you can call this like this from a python script:

import turkey

print turkey.do_something()

Check this: http://docs.python.org/2/c-api/arg.html for how to format the tuples and this.. http://docs.python.org/3.1/c-api/capsule.html for the doco on capsules

Answer 2

Assuming that you want some backwards compatibility, you want to use PyCObjects: http://docs.python.org/c-api/cobject.html

If you only want to use python 3, you can use a PyCapsule: http://docs.python.org/c-api/capsule.html

Basically the PyCObject stuff converts your opaque pointer into a PyObject that you can pass around within python and when you get back into C with one, you can unwrap it and use it.

PyCapsules are pretty similar except they add a few features, the main one being that they allow you to store multiple pointers in the Capsule, so it's basically a dict.

In your specific case, where you add the pointer, you'll just want to do this (error checking and destruction code removed):

PyObject *pystate = PyCObject_FromVoidPtr(State, NULL);
PyObject *dict = PyModule_GetDict(module);
PyDict_SetItemString(dict, "CStateObject", pystate);

# To retrieve it from self (assuming that it is an object)
PyObject *pystate = PyObject_GetAttrString(self, "CStateObject");
State *state = (State *)PyCObject_AsVoidPtr(pystate);