What Is The Cleanest Way to Call A Python Function From C++ with a SWIG Wrapped Object

Question

I have the following code, which implements a simple C++ class (ObjWithPyCallback) with a Python callback function. The idea is to call the Python function with "this" as the single argument.

The problem is that since ObjWithPyCallback is a SWIG wrapped object I need the SWIG typeinfo in order to create a Python object.

The problem with this is that it's inside of the SWIG generated file "ObjWithPyCallback_wrap.cxx". Can SWIG generate a header file? I have thus far not been able to make this happen.

However, even with a header file there is a circular dependency between SWIG and my main implementation, which is annoying. I'd like to find a way to avoid it if at all possible. Ultimately ObjWithPyCallback ends up in a different shared library than the Python bindings.

Is there a clean way to pull this off? I'm aware of this post, but it only addresses the mechanics of SWIG_NewPointerObj.

Thanks in advance for any help!

Here's the code:

File: example.py

import cb

def foo(x=None):
    print("Hello from Foo!")
    # I'd like x to be a reference to a ObjWithPyCallback object.
    print(x)

o = cb.ObjWithPyCallback()
o.setCallback(foo)
o.call()

File: ObjWithPyCallback.h

#include <Python.h>

class ObjWithPyCallback 
{
   public:

      ObjWithPyCallback();
      void setCallback(PyObject *callback);
      void call();

      PyObject *callback_;
};

File: ObjWithCallback.cpp

#include "ObjWithPyCallback.h"

#include <iostream>

ObjWithPyCallback::ObjWithPyCallback() : callback_(NULL) {}

void ObjWithPyCallback::setCallback(PyObject* callback)
{
   if (!PyCallable_Check(callback))
   {
      std::cerr << "Object is not callable.\n";
   }
   else
   {
      if ( callback_ ) Py_XDECREF(callback_);
      callback_ = callback;
      Py_XINCREF(callback_);
   }
}

void ObjWithPyCallback::call()
{
   if ( ! callback_ )
   {
      std::cerr << "No callback is set.\n";
   }
   else
   {
      // I want to call "callback_(*this)", how to do this cleanly?
      PyObject *result = PyObject_CallFunction(callback_, "");
      if (result == NULL)
         std::cerr << "Callback call failed.\n";
      else
         Py_DECREF(result);
   }
}

File:: ObjWithPyCallback.i

%module cb
%{
   #include "ObjWithPyCallback.h"
%}

%include "ObjWithPyCallback.h"

Answer 1

Below is my working solution for solving this problem. It uses the suggestions from both @omnifarious and @flexo above.

In particular we create a Callback class with a SWIG director and then derive from it in Python to get the required callback functionality without introducing a circular dependency.

In addition we provide an interface which allows any Python object that is callable to act as a callback. We achieve this by using the "pythonprend" directive in SWIG to prepend some code to the "setCallback" function. This code simply checks for a callable object and if it finds one, wraps it in an instance of a Callback.

Finally we deal with the memory issues related to having a C++ class (ObjWithPyCallback) reference a director object (i.e. a subclass of Callback).

File example.py:

import cb

class CB(cb.Callback):
    def __init__(self):
        super(CB, self).__init__()
    def call(self, x):
        print("Hello from CB!")
        print(x)

def foo(x):
    print("Hello from foo!")
    print(x)

class Bar:
    def __call__(self, x):
        print("Hello from Bar!")
        print(x)


o = cb.ObjWithPyCallback()
mycb=CB()
o.setCallback(mycb)
o.call()
o.setCallback(foo)
o.call()
o.setCallback(Bar())
o.call()

File ObjWithPyCallback.i:

%module(directors="1") cb
%{
   #include "Callback.h"
   #include "ObjWithPyCallback.h"
%}
%feature("director") Callback;
%feature("nodirector") ObjWithPyCallback;

%feature("pythonprepend") ObjWithPyCallback::setCallback(Callback&) %{
   if len(args) == 1 and (not isinstance(args[0], Callback) and callable(args[0])):
      class CallableWrapper(Callback):
         def __init__(self, f):
            super(CallableWrapper, self).__init__()
            self.f_ = f
         def call(self, obj):
            self.f_(obj)

      args = tuple([CallableWrapper(args[0])])
      args[0].__disown__()
   elif len(args) == 1 and isinstance(args[0], Callback):
      args[0].__disown__()


%}

%include "Callback.h"
%include "ObjWithPyCallback.h"

File Callback.h:

#ifndef CALLBACK_H
#define CALLBACK_H

class ObjWithPyCallback;

class Callback
{
   public:
      Callback(){}

      virtual ~Callback(){}
      virtual void call(ObjWithPyCallback& object){} 
};

#endif

File ObjWithPyCallback.h:

#ifndef OBJWITHPYCALLBACK_H
#define OBJWITHPYCALLBACK_H

class Callback;

class ObjWithPyCallback 
{
   public:

      ObjWithPyCallback();
      ~ObjWithPyCallback();
      void setCallback(Callback &callback);
      void call();

   private:

      Callback* callback_;
};

#endif

File ObjWithPyCallback.cpp:

#include "ObjWithPyCallback.h"
#include "Callback.h"

#include <iostream>

ObjWithPyCallback::ObjWithPyCallback() : callback_(NULL) {}

ObjWithPyCallback::~ObjWithPyCallback()
{
}

void ObjWithPyCallback::setCallback(Callback &callback)
{
   callback_ = &callback;
}

void ObjWithPyCallback::call()
{
   if ( ! callback_ )
   {
      std::cerr << "No callback is set.\n";
   }
   else
   {
      callback_->call(*this);
   }
}

Answer 2

1. General idea of solve the problem:

(1). Define a C++ class named Callback, which has a method run().

(2). Inherit Callback in Python code, and create a instance.

(3). Use C++ method to bind the instance to C++ pointor.

(4). Use the pointor to access run(), which is defined in python code.

2. Sample code

(1). example.h

class Callback{
    public:
    virtual void run(int n);                                                                                                                                                      
    virtual ~Callback() {}; 
};   
extern Callback * callback;
extern void doSomeWithCallback();
extern void setCallback(Callback * cb);

(2). example.cxx

#include <iostream>
#include "example.h"

int n=0;
Callback * callback = NULL;

void Callback::run(int n){ 
    std::cout << "This print from C++: n = " << n << std::endl;
}    

void setCallback(Callback * cb){
    callback = cb; 
}    

void doSomeWithCallback(){
    if(callback == NULL){
        std::cout << "Must set callback first!" << std::endl;
    }else{
        callback->run(n++);
    }                                                                                                                                                                                         
}

(3). example.i

/* File : example.i */                                                                                                                                                                        
%module(directors="1") example
%{
#include "example.h"                                                                                                                                                                          
%}

/* turn on director wrapping Callback */
%feature("director") Callback;

%include "example.h"

3. Compile

$ swig -c++ -python example.i
$ g++ -c -fPIC example.cxx example_wrap.cxx -I/usr/include/python2.7/
$ g++ -shared example.o example_wrap.o -o _example.so

4. Use in python shell

In [1]: import example

In [2]: example.doSomeWithCallback()
Must set callback first!

In [3]: callback = example.Callback()

In [4]: example.setCallback(callback)

In [5]: example.doSomeWithCallback()
This print from C++: n = 0

In [6]: class Callback(example.Callback):
   ...:     def run(self, n):
   ...:         print 'This print from Python: n =', n
   ...:         

In [7]: callback =  Callback()

In [8]: example.setCallback(callback)

In [9]: example.doSomeWithCallback()
This print from Python: n = 1

5. Other

I think there's more thing you need. Try:

$ ls swig-x.x.x/Examples/python

Answer 3

I would use SWIG's mechanisms for handling inheritance and have a callback class with a virtual function void call(). Then you use SWIG to enable that class to be derived from in Python.

In Python, you simply make sure that where the callback is set, you wrap it in an instance of a Python class derived from the C++ callback class, and make it's call member function execute the callback. That's also where you'd do the test to see if it's callable. Then you would call the setCallback function with this wrapper object.