How can I get methods to work as callbacks with python ctypes?

Question

I have an C api that i'm interfacing with the python ctypes package. Everything works well, except this little tidbit.

To register functions as callbacks to some notifications, i call this function :

void RegisterNotifyCallback( int loginId, int extraFlags, void *(*callbackFunc)(Notification *))

so in python my code looks like :

CALLBACK = ctypes.CFUNCTYPE(None, ctypes.POINTER(Notification))
func = CALLBACK(myPythonCallback)
myLib.RegisterNofityCallback(45454, 0, func)

If the value of myPythonCallback is a FunctionType (i.e. not a method), it works fine and calls the callback function with the correct parameters. If it's a MethodType, it still calls the method but then blows up with a segfault once the method is finished.

edit

To clarify, when i say function type i mean that myPythonCallback is defined as a function,

def myPythonCallback(Notification):
    ...do something with the Notification object i get passed in

When i say it's a MethodType, i mean a class method :

class MyClass(object):
    def myPythonCallback(self, Notification):
        ...do something with Notification

It blows up on the second type.

Is there an easy away around this ? Or can someone offer me an explanation as to why this is happening ?

Many thanks, Al.

edit

Digging a bit further with GDB gives me this :

Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 1544567712 (LWP 5389)] 0x555ae962 in instancemethod_dealloc (im=0x558ed644) at Objects/classobject.c:2360 2360    Objects/classobject.c: No such file or directory.
        in Objects/classobject.c (gdb) backtrace
#0  0x555ae962 in instancemethod_dealloc (im=0x558ed644) at Objects/classobject.c:2360
#1  0x555f6a61 in tupledealloc (op=0x5592010c) at Objects/tupleobject.c:220
#2  0x555aeed1 in instancemethod_call (func=0x558db8ec, arg=0x5592010c, kw=0x0) at Objects/classobject.c:2579
#3  0x5559aedb in PyObject_Call (func=0x558db8ec, arg=0x5592c0ec, kw=0x0) at Objects/abstract.c:2529
#4  0x5563d5af in PyEval_CallObjectWithKeywords (func=0x558db8ec, arg=0x5592c0ec, kw=0x0) at Python/ceval.c:3881
#5  0x5559ae6a in PyObject_CallObject (o=0x0, a=0x0) at Objects/abstract.c:2517

and the value passed into instancemethod_dealloc is :

(gdb) x 0x558ed644 0x558ed644:     0x00000000

Do you reckon this is fixable ?

Answer 1

You can't, so far as I know, call a bound method because it is missing the self parameter. I solve this problem using a closure, like this:

CALLBACK = ctypes.CFUNCTYPE(None, ctypes.POINTER(Notification))

class MyClass(object):

    def getCallbackFunc(self):
        def func(Notification):
            self.doSomething(Notification)
        return CALLBACK(func)

    def doRegister(self):
        myLib.RegisterNofityCallback(45454, 0, self.getCallbackFunc())

Answer 2

Its been said but the segfault is probably caused by the garbage collection of the method, you have to store the reference. I solved both the missing self parameters and the garbage collection problem like this.

CALLBACK = ctypes.CFUNCTYPE(None, ctypes.POINTER(Notification))

class MyClass(object):
    def myPythonCallback(self, Notification):
        ...do something with Notification
    
    def __init__(self):        
        self.myPythonCallback = CALLBACK(self.myPythonCallback)
        myLib.RegisterNofityCallback(45454, 0, self.myPythonCallback)

Every new instance of the class create the bound method (self.myPythonCallback), so each registered callback is specific to the class instance. We've held the reference to the callback in the class so it wont get garbage collected. The CFUNCTIONTYPE callback is executed as per normal same as if it were a global function <function_CFUNCTYPE>, and its inner callback is the bound method <bound method myPythoncCallback of MyClass 0xfffabca0>. Bound methods execute with self.

Note: although it looks a lot like you could just use @decorator syntax, that won't work. Decorators on methods decorate the original method and then bind to the class, so the effect is self.myPythonCallback would be a <bound CFUNCTYPE on MyClass> calling a plain method <function myPythonCallback>

The closeure solution provided by @David-Heffernan was my inspiration,

Answer 3

This may be a ctypes bug dealing with the magic behind setting up the stack for methods vs functions.

Have you tried using a wrapper via lambda or function?

Lambda:

func = CALLBACK(lambda x: myPythonCallback(x))

Function:

def wrapper(x): myPythonCallback(x)
func = CALLBACK(wrapper)

There's a third method using closures. If you're setting up the callback from inside the class, the method defined below should inherit the "self" argument due to scope - making it act like a class method despite being a regular function.

class Foo:
    def __init__(self):
        def myPythonCallback(Notification):
            print self # it's there!
            pass # do something with notification

        func = CALLBACK(myPythonCallback)
        myLib.RegisterNofityCallback(45454, 0, func)