Python ctypes definition for c struct

Question

I am trying to call some c code generated by the Matlab coder. Matlab uses a c struct called emxArray to represent matrices (documented here: http://www.mathworks.co.uk/help/fixedpoint/ug/c-code-interface-for-unbounded-arrays-and-structure-fields.html).

struct emxArray_real_T
{
    double *data;
    int *size;
    int allocatedSize;
    int numDimensions;
    boolean_T canFreeData;
};

I have little experience ctypes and am struggling to create an equivalent struct that I can then use to pass vectors back and forth to the functions defined in the c .so

Here is where I have got so far in python...

class EmxArray(ctypes.Structure):
    """ creates a struct to match emxArray_real_T """

    _fields_ = [('data', ctypes.POINTER(ctypes.c_double)),
                ('size', ctypes.POINTER(ctypes.c_int)),
                ('allocatedSize', ctypes.c_int),
                ('numDimensions', ctypes.c_int),
                ('canFreeData', ctypes.c_bool)]    

However if I define this:

data = (1.1, 1.2, 1.3, 1.4)
L = len(data)

x = EmxArray()
x.data = (ctypes.c_double * L)(*data)
x.data = (ctypes.c_int * 1)(L)    

this then works

print len(x.data[:L]) 

for v in x.data[:L]: print v

Edit: I have tidied up and adopted Roland's suggestion and can extract the data using

data_out = x.data[:L]

I need to investigate further to see if I can successfully use this struct to pass and receive data from the c code.

Solution

Implementing the ctypes struct as suggested by Roland didn't work - the returned values were garbage, I never worked out why as I pursued a python based implementation of lilbil's answer. I've accepted that answer as it was closest...

I'll document my solution here as it might save someone else wasting as much time as I have.

Firstly I've generated a simple matlab function that multiplies each element of a function by itself & used the coder to compile this to a c .so. This is imported to python using ctypes. The code is as follows...

import ctypes

LIBTEST = '..../dll/emx_test/'
EMX = ctypes.cdll.LoadLibrary(LIBTEST + 'emx_test.so')
init = EMX.emx_test_initialize()

# Create a data structure to hold the pointer generated by emxCreateWrapper...
class Opaque(ctypes.Structure):
    pass

# make some random data to pass in
data_in = [1., 2., 4., 8., 16.]
L = len(data_in)
# create an empty array of the same size for the output
data_ou = [0] * L

# put this in a ctypes array
ina = (ctypes.c_double * L)(*data_in)
oua = (ctypes.c_double * L)(*data_ou)
# create a pointer for these arrays & set the rows and columns of the matrix
inp = ctypes.pointer(ina)
oup = ctypes.pointer(oua)

nrows = ctypes.c_int(1)
ncols = ctypes.c_int(L)

# use EMX.emxCreateWrapper_real_T(double *data, int rows, int cols) to generate an emx wrapping the data 
# input arg types are a pointer to the data NOTE its not great to have to resize the ctypes.c_double but cant see another way
EMX.emxCreateWrapper_real_T.argtypes = [ctypes.POINTER(ctypes.c_double * L), ctypes.c_int, ctypes.c_int]
# a pointer to the emxArray is returned and stored in Opaque
EMX.emxCreateWrapper_real_T.restype = ctypes.POINTER(Opaque)
# use emxCreateWrapper
in_emx = EMX.emxCreateWrapper_real_T(inp, nrows, ncols)
ou_emx = EMX.emxCreateWrapper_real_T(oup, nrows, ncols)

# so now we have to emx's created and have pointers to them we can run the emx_test
# emx test looks like this in matlab
#
# function res = emx_test ( in )
#     res = in .* in;
# end
#
# so basically it multiplies each element of the matrix by itself
# 
# therefore [1., 2., 4., 8., 16.] should become [1., 4., 8., 64., 256.]

EMX.emx_test(in_emx, ou_emx)

# and voila...that's what we get
print 'In: ', ina[:L]
print 'Out:', oua[:L]

Output:

In: [1.0, 2.0, 4.0, 8.0, 16.0]
Out:[1.0, 4.0, 16.0, 64.0, 256.0]

Thank you to everyone for your time & suggestions.

Answer 1

Just create a pointer, assign the data afterwards;

import ctypes

class EmxArray(ctypes.Structure):
    """ creates a struct to match emxArray_real_T """

    _fields_ = [('data', ctypes.POINTER(ctypes.c_double)),
                ('size', ctypes.POINTER(ctypes.c_int)),
                ('allocatedSize', ctypes.c_int),
                ('numDimensions', ctypes.c_int),
                ('canFreeData', ctypes.c_bool)]

data = (1.3, 3.5, 2.7, 4.1)
L = len(data)

e = EmxArray()
e.data = (ctypes.c_double * L)(*data)
e.size = (ctypes.c_int * 1)(L)
# et cetera

Answer 2

I'm not well-versed with the Python-C interface, so what I'm suggesting may be less than ideal. My guess is that likely the crash is because x->data is never initialized and the memory to which it is pointing isn't allocated.

An approach I have taken when interfacing to MATLAB Coder generated code from other languages in the presence of emxArray arguments is to hand-write a C interface function that provides a simpler API. This relieves the burden of needing to construct an emxArray in the other environment (Android Java in my particular case). If the generated function foo takes and returns a 2-D double array, then something like the following could work:

void foo(double *x, int *szx, double **y, int *szy);

This function would take a pointer to the input data and its size and provide a pointer to the output data and its size. The implementation would look something like:

void foo(double *x, int *szx, double **y, int *szy) 
{
  emxArray_real_T *pEmx;
  emxArray_real_T *pEmy;

  /* Create input emxArray assuming 2-dimensional input */
  pEmx = emxCreateWrapper_real_T(x, szx[0], szx[1]);

  /* Create output emxArray (assumes that the output is not */
  /* written before allocation occurs) assuming 2-D output  */
  pEmy = emxCreateWrapper_real_T(NULL, 0, 0);

  /* Call generated code (call foobar_initialize/terminate elsewhere) */
  foobar(pEmx, pEmy);

  /* Unpack result - You may want to MALLOC storage in *y and */
  /* MEMCPY there alternatively                               */
  *y = pEmy->data;
  szy[0] = pEmy->size[0];
  szy[1] = pEmy->size[1];

  /* Clean up any memory allocated in the emxArrays (e.g. the size vectors) */
  emxDestroyArray_real_T(pEmx);
  emxDestroyArray_real_T(pEmy);
}

You should be able to call this function from Python more simply and pass in the desired data as needed.

My other answer has more details on the emxArray_* functions found in the file foobar_emxAPI.h.