How do you wrap a C function that returns a pointer to a malloc'd array with ctypes?

Question

I have a C function that reads a binary file and returns a dynamically sized array of unsigned integers (the size is based off metadata from the binary file):

//example.c
#include <stdio.h>
#include <stdlib.h>

__declspec(dllexport)unsigned int *read_data(char *filename, size_t* array_size){
  FILE *f = fopen(filename, "rb");
  fread(array_size, sizeof(size_t), 1, f);
  unsigned int *array = (unsigned int *)malloc(*array_size * sizeof(unsigned int));
  fread(array, sizeof(unsigned int), *array_size, f);
  fclose(f);

  return array;
}

This answer appears to be saying that the correct way to pass the created array from C to Python is something like this:

# example_wrap.py
from ctypes import *
import os

os.add_dll_directory(os.getcwd())
indexer_dll = CDLL("example.dll")

def read_data(filename):
    filename = bytes(filename, 'utf-8')
    size = c_size_t()
    ptr = indexer_dll.read_data(filename, byref(size))
    return ptr[:size]

However, when I run the python wrapper, the code silently fails at ptr[:size] as if I'm trying to access an array out of bounds, and I probably am, but what is the correct way to pass this dynamically size array?

Answer 1

A few considerations:

First, you need to properly set the prototype of the C function so that ctypes can properly convert between the C and Python types.

Second, since size is actually a ctypes.c_size_t object, you actually need to use size.value to access the numeric value of the array size.

Third, since ptr[:size.value] actually copies the array contents to a Python list, you'll want to make sure you also free() the allocated C array since you're not going to use it anymore.

(Perhaps copying the array to a Python list is not ideal here, but I'll assume it's ok here, since otherwise you have more complexity in handling the C array in Python.)

This should work:

from ctypes import *
import os

os.add_dll_directory(os.getcwd())
indexer_dll = CDLL("example.dll")
indexer_dll.read_data.argtypes = [c_char_p, POINTER(c_size_t)
indexer_dll.read_data.restype = POINTER(c_int)
libc = cdll.msvcrt

def read_data(filename):
    filename = bytes(filename, 'utf-8')
    size = c_size_t()
    ptr = indexer_dll.read_data(filename, byref(size))
    result = ptr[:size.value]
    libc.free(ptr)
    return result