How do I prevent python deallocating ctypes callbacks at exit-time?

Question

Suppose I have the following shared library to be loaded by ctypes. It allows you to register a callback to be invoked when the program exits, or when you invoke it yourself:

#include <stdlib.h>
static void (*callback)(void);

void invoke_callback(void)
{
    callback();
}

void set_callback(void (*new_callback)(void))
{
    callback = new_callback;
}

void init(void)
{
    atexit(invoke_callback);
}

Then suppose I load this library via the magic of ctypes:

import ctypes
shared = ctypes.CDLL('./test.so')

#a callback function
def callback():
    print "callback invoked"

#register functions to run at exit
shared.init()
#set the callback function to invoke
shared.set_callback(ctypes.CFUNCTYPE(None)(callback))
#invoke the callback function
shared.invoke_callback()

#...callback also invoked here, right?

I expected the output of this to be something like the following:

callback invoked
callback invoked

Unfortunately for me, it looked a bit more like this:

callback invoked
Segmentation fault

Why is this, you ask? Well, it would seem that by the time the atexit functions are called, the python interpreter has de-allocated the memory previously containing the callbacks:

(gdb) backtrace
#0  0x000000000049b11d in ?? () <- uh-oh
#1  0x000000000046d245 in ?? () <- ctypes' wrapper?
#2  0x00007ffff6b554a9 in ?? () <- ctypes
   from /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so
#3  0x00007ffff6944baf in ffi_closure_unix64_inner ()
   from /usr/lib/x86_64-linux-gnu/libffi.so.6
#4  0x00007ffff6944f28 in ffi_closure_unix64 ()
   from /usr/lib/x86_64-linux-gnu/libffi.so.6
#5  0x00007ffff673e71d in invoke_callback () at test.c:6 <- me
#6  0x00007ffff6f2abc9 in __run_exit_handlers (status=0, 
    listp=0x7ffff72965a8 <__exit_funcs>, 
    run_list_atexit=run_list_atexit@entry=true) at exit.c:82
#7  0x00007ffff6f2ac15 in __GI_exit (status=<optimized out>) at exit.c:104
#8  0x00007ffff6f14b4c in __libc_start_main (main=0x497d80 <main>, argc=2, 
    argv=0x7fffffffe408, init=<optimized out>, fini=<optimized out>, 
    rtld_fini=<optimized out>, stack_end=0x7fffffffe3f8) at libc-start.c:321
#9  0x0000000000497ca0 in _start ()

Now, my question. I'm actually trying to bind to a largish C codebase (which I can't modify) containing several callbacks called at exit-time. These are currently causing segmentation faults when the test program exits. Is it possible to prevent this happening?

Answer 1

I might be late to the party here but I encountered a similar issue recently. Basically, the callback is being garbage collected by Python. If you do this:

callback_type = ctypes.CFUNCTYPE(None)
wrapped_callback = callback_type(callback)
shared.set_callback(wrapped_callback)

it should solve the segfault.