How to use NumPy array with ctypes?

Question

I am still writing on a python interface for my c code with ctypes. Today I substituted my file reading function with a python version, which was programmed by somebody else using NumPy. The 'old' c version was called with a byref(p_data) while p_data=PFloat() (see below). The main function takes the p_data.

Old file reading:

p_data=POINTER(c_float)
foo.read(filename,byref(p_data))
result=foo.pymain(p_data)

The python file reading function, on the other hand, returns a NumPy array. My question now is:

How do I convert a NumPy array to POINTER(c_float)?

I googled but only found the other way around: C arrays through ctypes accessed as NumPy arrays and things I didn't understand: C-Types Foreign Function Interface (numpy.ctypeslib)

[update] corrected a mistake in the example code

Answer 1

Your code looks like it has some confusion in it -- ctypes.POINTER() creates a new ctypes pointer class, not a ctypes instance. Anyway, the easiest way to pass a NumPy array to ctypes code is to use the numpy.ndarray's ctypes attribute's data_as method. Just make sure the underlying data is the right type first. For example:

import ctypes
import numpy
c_float_p = ctypes.POINTER(ctypes.c_float)
data = numpy.array([[0.1, 0.1], [0.2, 0.2], [0.3, 0.3]])
data = data.astype(numpy.float32)
data_p = data.ctypes.data_as(c_float_p)

Answer 2

Using np.ndarrays as ctypes arguments

The preferable approach is using ndpointer, as mentioned in the numpy-docs.

This approach is more flexible than using, for example, POINTER(c_double), since several restrictions can be specified, which are verified upon calling the ctypes function. These include data type, number of dimensions, shape and flags. If a given array does not satisfy the specified restrictions, a TypeError is raised.

Minimal, Reproducible Example

Calling memcpy from python. Eventually the filename of the standard C-library libc.so.6 needs to be adjusted.

import ctypes
import numpy as np

n_bytes_f64 = 8
nrows = 2
ncols = 5

clib = ctypes.cdll.LoadLibrary("libc.so.6")

clib.memcpy.argtypes = [
    np.ctypeslib.ndpointer(dtype=np.float64, ndim=2, flags='C_CONTIGUOUS'),
    np.ctypeslib.ndpointer(dtype=np.float64, ndim=1, flags='C_CONTIGUOUS'),
    ctypes.c_size_t]
clib.memcpy.restype = ctypes.c_void_p

arr_from = np.arange(nrows * ncols).astype(np.float64)
arr_to = np.empty(shape=(nrows, ncols), dtype=np.float64)

print('arr_from:', arr_from)
print('arr_to:', arr_to)

print('\ncalling clib.memcpy ...\n')
clib.memcpy(arr_to, arr_from, nrows * ncols * n_bytes_f64)

print('arr_from:', arr_from)
print('arr_to:', arr_to)

Output

arr_from: [0. 1. 2. 3. 4. 5. 6. 7. 8. 9.]
arr_to: [[0.0e+000 4.9e-324 9.9e-324 1.5e-323 2.0e-323]
 [2.5e-323 3.0e-323 3.5e-323 4.0e-323 4.4e-323]]

calling clib.memcpy ...

arr_from: [0. 1. 2. 3. 4. 5. 6. 7. 8. 9.]
arr_to: [[0. 1. 2. 3. 4.]
 [5. 6. 7. 8. 9.]]

If you modify the ndim=1/2 arguments of ndpointer to be inconsistent with the dimensions of arr_from/arr_to, the code fails with an ArgumentError.

As the title of this question is quite general, ...

Constructing a np.ndarray from a ctypes.c_void_p result

Minimal, Reproducible Example

In the following example, some memory is allocated by malloc and filled with 0s by memset. Then a numpy array is constructed, to access this memory. Of course the occur some ownership issues, as python will not free memory, which was allocated in c. To avoid memory leaks, one has to free the allocated memory again by ctypes. The copy method can be used for the np.ndarray to acquire ownership.

import ctypes
import numpy as np

n_bytes_int = 4
size = 7

clib = ctypes.cdll.LoadLibrary("libc.so.6")

clib.malloc.argtypes = [ctypes.c_size_t]
clib.malloc.restype = ctypes.c_void_p

clib.memset.argtypes = [
    ctypes.c_void_p,
    ctypes.c_int,
    ctypes.c_size_t]
clib.memset.restype = np.ctypeslib.ndpointer(
    dtype=np.int32, ndim=1, flags='C_CONTIGUOUS')

clib.free.argtypes = [ctypes.c_void_p]
clib.free.restype = ctypes.c_void_p


pntr = clib.malloc(size * n_bytes_int)
ndpntr = clib.memset(pntr, 0, size * n_bytes_int)
print(type(ndpntr))
ctypes_pntr = ctypes.cast(ndpntr, ctypes.POINTER(ctypes.c_int))
print(type(ctypes_pntr))
print()
arr_noowner = np.ctypeslib.as_array(ctypes_pntr, shape=(size,))
arr_owner = np.ctypeslib.as_array(ctypes_pntr, shape=(size,)).copy()
# arr_owner = arr_noowner.copy()


print('arr_noowner (at {:}): {:}'.format(arr_noowner.ctypes.data, arr_noowner))
print('arr_owner (at {:}): {:}'.format(arr_owner.ctypes.data, arr_owner))

print('\nfree allocated memory again ...\n')
_ = clib.free(pntr)

print('arr_noowner (at {:}): {:}'.format(arr_noowner.ctypes.data, arr_noowner))
print('arr_owner (at {:}): {:}'.format(arr_owner.ctypes.data, arr_owner))

print('\njust for fun: free some python-memory ...\n')
_ = clib.free(arr_owner.ctypes.data_as(ctypes.c_void_p))

print('arr_noowner (at {:}): {:}'.format(arr_noowner.ctypes.data, arr_noowner))
print('arr_owner (at {:}): {:}'.format(arr_owner.ctypes.data, arr_owner))

Output

<class 'numpy.ctypeslib.ndpointer_<i4_1d_C_CONTIGUOUS'>
<class '__main__.LP_c_int'>

arr_noowner (at 104719884831376): [0 0 0 0 0 0 0]
arr_owner (at 104719884827744): [0 0 0 0 0 0 0]

free allocated memory again ...

arr_noowner (at 104719884831376): [ -7687536     24381 -28516336     24381         0         0         0]
arr_owner (at 104719884827744): [0 0 0 0 0 0 0]

just for fun: free some python-memory ...

arr_noowner (at 104719884831376): [ -7687536     24381 -28516336     24381         0         0         0]
arr_owner (at 104719884827744): [ -7779696     24381 -28516336     24381         0         0         0]