How can I implement a C++ class in Python, to be called by C++?

Question

I have a class interface written in C++. I have a few classes that implement this interface also written in C++. These are called in the context of a larger C++ program, which essentially implements "main". I want to be able to write implementations of this interface in Python, and allow them to be used in the context of the larger C++ program, as if they had been just written in C++.

There's been a lot written about interfacing python and C++ but I cannot quite figure out how to do what I want. The closest I can find is here: http://www.cs.brown.edu/~jwicks/boost/libs/python/doc/tutorial/doc/html/python/exposing.html#python.class_virtual_functions, but this isn't quite right.

To be more concrete, suppose I have an existing C++ interface defined something like:

// myif.h class myif {    public:      virtual float myfunc(float a); }; 

What I want to be able to do is something like:

// mycl.py ... some magic python stuff ... class MyCl(myif):   def myfunc(a):     return a*2 

Then, back in my C++ code, I want to be able to say something like:

// mymain.cc void main(...) {   ... some magic c++ stuff ...   myif c = MyCl();  // get the python class   cout << c.myfunc(5) << endl;  // should print 10 } 

I hope this is sufficiently clear ;)

Answer 1

There's two parts to this answer. First you need to expose your interface in Python in a way which allows Python implementations to override parts of it at will. Then you need to show your C++ program (in main how to call Python.

---

Exposing the existing interface to Python:

The first part is pretty easy to do with SWIG. I modified your example scenario slightly to fix a few issues and added an extra function for testing:

// myif.h class myif {    public:      virtual float myfunc(float a) = 0; };  inline void runCode(myif *inst) {   std::cout << inst->myfunc(5) << std::endl; } 

For now I'll look at the problem without embedding Python in your application, i.e. you start excetion in Python, not in int main() in C++. It's fairly straightforward to add that later though.

First up is getting cross-language polymorphism working:

%module(directors="1") module  // We need to include myif.h in the SWIG generated C++ file %{ #include <iostream> #include "myif.h" %}  // Enable cross-language polymorphism in the SWIG wrapper.  // It's pretty slow so not enable by default %feature("director") myif;  // Tell swig to wrap everything in myif.h %include "myif.h" 

To do that we've enabled SWIG's director feature globally and specifically for our interface. The rest of it is pretty standard SWIG though.

I wrote a test Python implementation:

import module  class MyCl(module.myif):   def __init__(self):     module.myif.__init__(self)   def myfunc(self,a):     return a*2.0  cl = MyCl()  print cl.myfunc(100.0)  module.runCode(cl) 

With that I was then able to compile and run this:

 swig -python  -c++ -Wall myif.i  g++ -Wall -Wextra -shared -o _module.so myif_wrap.cxx -I/usr/include/python2.7 -lpython2.7  python mycl.py  200.0 10 

Exactly what you'd hope to see from that test.

---

Embedding the Python in the application:

Next up we need to implement a real version of your mymain.cc. I've put together a sketch of what it might look like:

#include <iostream> #include "myif.h" #include <Python.h>  int main() {   Py_Initialize();    const double input = 5.0;    PyObject *main = PyImport_AddModule("__main__");   PyObject *dict = PyModule_GetDict(main);   PySys_SetPath(".");   PyObject *module = PyImport_Import(PyString_FromString("mycl"));   PyModule_AddObject(main, "mycl", module);    PyObject *instance = PyRun_String("mycl.MyCl()", Py_eval_input, dict, dict);   PyObject *result = PyObject_CallMethod(instance, "myfunc", (char *)"(O)" ,PyFloat_FromDouble(input));    PyObject *error = PyErr_Occurred();   if (error) {     std::cerr << "Error occured in PyRun_String" << std::endl;     PyErr_Print();   }    double ret = PyFloat_AsDouble(result);   std::cout << ret << std::endl;    Py_Finalize();   return 0; } 

It's basically just standard embedding Python in another application. It works and gives exactly what you'd hope to see also:

 g++ -Wall -Wextra -I/usr/include/python2.7 main.cc -o main -lpython2.7 ./main 200.0 10 10 

---

The final piece of the puzzle is being able to convert the PyObject* that you get from creating the instance in Python into a myif *. SWIG again makes this reasonably straightforward.

First we need to ask SWIG to expose its runtime in a headerfile for us. We do this with an extra call to SWIG:

 swig -Wall -c++ -python -external-runtime runtime.h 

Next we need to re-compile our SWIG module, explicitly giving the table of types SWIG knows about a name so we can look it up from within our main.cc. We recompile the .so using:

 g++ -DSWIG_TYPE_TABLE=myif -Wall -Wextra -shared -o _module.so myif_wrap.cxx -I/usr/include/python2.7 -lpython2.7 

Then we add a helper function for converting the PyObject* to myif* in our main.cc:

#include "runtime.h" // runtime.h was generated by SWIG for us with the second call we made  myif *python2interface(PyObject *obj) {   void *argp1 = 0;   swig_type_info * pTypeInfo = SWIG_TypeQuery("myif *");    const int res = SWIG_ConvertPtr(obj, &argp1,pTypeInfo, 0);   if (!SWIG_IsOK(res)) {     abort();   }   return reinterpret_cast<myif*>(argp1); } 

Now this is in place we can use it from within main():

int main() {   Py_Initialize();    const double input = 5.5;    PySys_SetPath(".");   PyObject *module = PyImport_ImportModule("mycl");    PyObject *cls = PyObject_GetAttrString(module, "MyCl");   PyObject *instance = PyObject_CallFunctionObjArgs(cls, NULL);    myif *inst = python2interface(instance);   std::cout << inst->myfunc(input) << std::endl;    Py_XDECREF(instance);   Py_XDECREF(cls);    Py_Finalize();   return 0; } 

Finally we have to compile main.cc with -DSWIG_TYPE_TABLE=myif and this gives:

 ./main 11