How to access arrays passed to ctypes callbacks as numpy arrays?

Question

I'm trying to integrate some numerical code written in C into a Python library using numpy and ctypes. I've already got the actual computations working but would now like to report the progress of the intermediate steps of my algorithm to a callback function in my Python code. While I can call the callback function successfully, I'm not able to retrieve the data in the x array passed to the callback. In the callback, x is an ndpointer object that I can't seem to dereference.

Current Code

Consider this minimal example:

test.h:

typedef void (*callback_t)(
    double *x,
    int n
);

void callback_test(double* x, int n, callback_t callback);

test.c:

#include "test.h"

void callback_test(double* x, int n, callback_t callback) {
    for(int i = 1; i <= 5; i++) {

        for(int j = 0; j < n; j++) {
            x[j] = x[j] / i;
        }

        callback(x, n);
    }
}

test.py:

#!/usr/bin/env python

import numpy as np
import numpy.ctypeslib as npct
import ctypes
import os.path

array_1d_double = npct.ndpointer(dtype=np.double, ndim=1, flags='CONTIGUOUS')

callback_func = ctypes.CFUNCTYPE(
    None,            # return
    array_1d_double, # x
    ctypes.c_int     # n
)

libtest = npct.load_library('libtest', os.path.dirname(__file__))
libtest.callback_test.restype = None
libtest.callback_test.argtypes = [array_1d_double, ctypes.c_int, callback_func]


@callback_func
def callback(x, n):
    print("x: {0}, n: {1}".format(x, n))


if __name__ == '__main__':
    x = np.array([20, 13, 8, 100, 1, 3], dtype=np.double)
    libtest.callback_test(x, x.shape[0], callback)

Current Output

After compiling and running the script, I get the following output:

x: <ndpointer_<f8_1d_CONTIGUOUS object at 0x7f9b55faba70>, n: 6
x: <ndpointer_<f8_1d_CONTIGUOUS object at 0x7f9b55faba70>, n: 6
x: <ndpointer_<f8_1d_CONTIGUOUS object at 0x7f9b55faba70>, n: 6
x: <ndpointer_<f8_1d_CONTIGUOUS object at 0x7f9b55faba70>, n: 6
x: <ndpointer_<f8_1d_CONTIGUOUS object at 0x7f9b55faba70>, n: 6

I've also tried the subsetting operator x[0:n] (TypeError: 'ndpointer_x.value (returns the pointer as a number).

Hackish Solution

If I use the following alternative definition of callback_func:

callback_func = ctypes.CFUNCTYPE(
    None,            # return
    ctypes.POINTER(ctypes.c_double), # x
    ctypes.c_int     # n
)

and the following alternative callback function:

@callback_func
def callback(x, n):
    print("x: {0}, n: {1}".format(x[:n], n))

I get the desired results:

x: [20.0, 13.0, 8.0, 100.0, 1.0, 3.0], n: 6
x: [10.0, 6.5, 4.0, 50.0, 0.5, 1.5], n: 6
x: [3.3333333333333335, 2.1666666666666665, 1.3333333333333333, 16.666666666666668, 0.16666666666666666, 0.5], n: 6
x: [0.8333333333333334, 0.5416666666666666, 0.3333333333333333, 4.166666666666667, 0.041666666666666664, 0.125], n: 6
x: [0.16666666666666669, 0.10833333333333332, 0.06666666666666667, 0.8333333333333334, 0.008333333333333333, 0.025], n: 6

My Question

Is there a more more numpy-ish way of accessing x in the callback? Rather than subscripting and then converting back to a numpy.array, I would prefer to access the data pointed to by the ndpointer since I would like to limit the amount of copies of x (and for the sake of elegant code)

I've uploaded a gist of the whole mini-example if you want to experiment on my code.

Answer 1

I've found a solution using ctypes.POINTER(ctypes.c_double) and numpy.ctypeslib.as_array - according to the numpy.ctypeslib documentation, this will share the memory with the array:

callback_func = ctypes.CFUNCTYPE(
    None,            # return
    ctypes.POINTER(ctypes.c_double), # x
    ctypes.c_int     # n
)

[...]

@callback_func
def callback(x, n):
    x = npct.as_array(x, (n,))
    print("x: {0}, n: {1}".format(x, n))

Anyone with a more elegant solution, perhaps using the ndpointer objects?