Mex files: how to return an already allocated matlab array

Question

I have found a really tricky problem, which I can not seem to fix easily. In short, I would like to return from a mex file an array, which has been passed as mex function input. You could trivially do this:

void mexFunction(int nargout, mxArray *pargout [ ], int nargin, const mxArray *pargin[])
{
   pargout[0] = pargin[0];
}

But this is not what I need. I would like to get the raw pointer from pargin[0], process it internally, and return a freshly created mex array by setting the corresponding data pointer. Like that:

#include <mex.h>

void mexFunction(int nargout, mxArray *pargout [ ], int nargin, const mxArray *pargin[])
{
  mxArray *outp;
  double *data;
  int m, n;

  /* get input array */
  data = mxGetData(pargin[0]);
  m = mxGetM(pargin[0]);
  n = mxGetN(pargin[0]);

  /* copy pointer to output array */
  outp = mxCreateNumericMatrix(0,0,mxDOUBLE_CLASS,mxREAL);
  mxSetM(outp, m);
  mxSetN(outp, n);
  mxSetData(outp, data);
  /* segfaults with or without the below line */
  mexMakeMemoryPersistent(data);
  pargout[0] = outp;
}

It doesn't work. I get a segfault, if not immediately, then after a few calls. I believe nothing is said about such scenario in the documentation. The only requirement is hat the data pointer has been allocated using mxCalloc, which it obviously has. Hence, I would assume this code is legal.

I need to do this, because I am parsing a complicated MATLAB structure into my internal C data structures. I process the data, some of the data gets re-allocated, some doesn't. I would like to transparently return the output structure, without thinking when I have to simply copy an mxArray (first code snippet), and when I actually have to create it.

Please help!

EDIT

After further looking and discussing with Amro, it seems that even my first code snippet is unsupported and can cause MATLAB crashes in certain situations, e.g., when passing structure fields or cell elements to such mex function:

>> a.field = [1 2 3];
>> b = pargin_to_pargout(a.field);   % ok - works and assigns [1 2 3] to b
>> pargin_to_pargout(a.field);       % bad - segfault

It seems I will have to go down the 'undocumented MATLAB' road and use mxCreateSharedDataCopy and mxUnshareArray.

Answer 1

You should use mxDuplicateArray, thats the documented way:

#include "mex.h"

void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[])
{
    plhs[0] = mxDuplicateArray(prhs[0]);
}

Answer 2

While undocumented, the MEX API function mxCreateSharedDataCopy iswas given as a solution by MathWorks, now apparently disavowed, for creating a shared-data copy of an mxArray. MathWorks even provides an example in their solution, mxsharedcopy.c.

As described in that removed MathWorks Solution (1-6NU359), the function can be used to clone the mxArray header. However, the difference between doing plhs[0] = prhs[0]; and plhs[0] = mxCreateSharedDataCopy(prhs[0]); is that the first version just copies the mxArray* (a pointer) and hence does not create a new mxArray container (at least not until the mexFunction returns and MATLAB works it's magic), which would increment the data's reference count in both mxArrays.

Why might this be a problem? If you use plhs[0] = prhs[0]; and make no further modification to plhs[0] before returning from mexFunction, all is well and you will have a shared data copy thanks to MATLAB. However, if after the above assignment you modify plhs[0] in the MEX function, the change be seen in prhs[0] as well since it refers to the same data buffer. On the other hand, when explicitly generating a shared copy (with mxCreateSharedDataCopy) there are two different mxArray objects and a change to one array's data will trigger a copy operation resulting in two completely independent arrays. Also, direct assignment can cause segmentation faults in some cases.

Modified MathWorks Example

Start with an example using a modified mxsharedcopy.c from the MathWorks solution referenced above. The first important step is to provide the prototype for the mxCreateSharedDataCopy function:

/* Add this declaration because it does not exist in the "mex.h" header */
extern mxArray *mxCreateSharedDataCopy(const mxArray *pr);

As the comment states, this is not in mex.h, so you have to declare this yourself.

The next part of the mxsharedcopy.c creates new mxArrays in the following ways:

1. A deep copy via mxDuplicateArray:

   copy1 = mxDuplicateArray(prhs[0]);
   

2. A shared copy via mxCreateSharedDataCopy:

   copy2 = mxCreateSharedDataCopy(copy1);
   

3. Direct copy of the mxArray*, added by me:

   copy0 = prhs[0]; // OK, but don't modify copy0 inside mexFunction!
   

Then it prints the address of the data buffer (pr) for each mxArray and their first values. Here is the output of the modified mxsharedcopy(x) for x=ones(1e3);:

prhs[0] = 72145590, mxGetPr = 18F90060, value = 1.000000
copy0   = 72145590, mxGetPr = 18F90060, value = 1.000000
copy1   = 721BF120, mxGetPr = 19740060, value = 1.000000
copy2   = 721BD4B0, mxGetPr = 19740060, value = 1.000000

What happened:

1. As expected, comparing prhs[0] and copy0 we have not created anything new except another pointer to the same mxArray.
2. Comparing prhs[0] and copy1, notice that mxDuplicateArray created a new mxArray at address 721BF120, and copied the data into a new buffer at 19740060.
3. copy2 has a different address (mxArray*) from copy1, meaning it is also a different mxArray not just the same one pointed to by different variables, but they both share the same data at address 19740060.

The question reduces to: Is it safe to return in plhs[0] either of copy0 or copy2 (from simple pointer copy or mxCreateSharedDataCopy, respectively) or is it necessary to use mxDuplicateArray, which actually copies the data? We can show that mxCreateSharedDataCopy would work by destroying copy1 and verifying that copy2 is still valid:

mxDestroyArray(copy1);
copy2val0 = *mxGetPr(copy2); % no crash!

Applying Shared-Data Copy to Input

Back to the question. Take this a step further than the MathWorks example and return a share-data copy of the input. Just do:

if (nlhs>0) plhs[0] = mxCreateSharedDataCopy(prhs[0]);

Hold your breath!

>> format debug
>> x=ones(1,2)
x =

Structure address = 9aff820     % mxArray*
m = 1
n = 2
pr = 2bcc8500                   % double*
pi = 0
     1     1
>> xDup = mxsharedcopy(x)
xDup =

Structure address = 9afe2b0     % mxArray* (different)
m = 1
n = 2
pr = 2bcc8500                   % double* (same)
pi = 0
     1     1
>> clear x
>> xDup % hold your breath!
xDup =

Structure address = 9afe2b0 
m = 1
n = 2
pr = 2bcc8500                    % double* (still same!)
pi = 0
     1     1

Now for a temporary input (without format debug):

>> tempDup = mxsharedcopy(2*ones(1e3));
>> tempDup(1)
ans =
     2

Interestingly, if I test without mxCreateSharedDataCopy (i.e. with just plhs[0] = prhs[0];), MATLAB doesn't crash but the output variable never materializes:

>> tempDup = mxsharedcopy(2*ones(1e3)) % no semi-colon
>> whos tempDup
>> tempDup(1)
Undefined function 'tempDup' for input arguments of type 'double'.

R2013b, Windows, 64-bit.

mxsharedcopy.cpp (modified C++ version):

#include "mex.h"

/* Add this declaration because it does not exist in the "mex.h" header */
extern "C" mxArray *mxCreateSharedDataCopy(const mxArray *pr);
bool mxUnshareArray(const mxArray *pr, const bool noDeepCopy); // true if not successful

void mexFunction(int nlhs,mxArray *plhs[],int nrhs,const mxArray *prhs[])
{
    mxArray *copy1(NULL), *copy2(NULL), *copy0(NULL);

    //(void) plhs; /* Unused parameter */

    /* Check for proper number of input and output arguments */
    if (nrhs != 1)
        mexErrMsgTxt("One input argument required.");
    if (nlhs > 1)
        mexErrMsgTxt("Too many output arguments.");

    copy0 = const_cast<mxArray*>(prhs[0]); // ADDED

    /* First make a regular deep copy of the input array */
    copy1 = mxDuplicateArray(prhs[0]);

    /* Then make a shared copy of the new array */
    copy2 = mxCreateSharedDataCopy(copy1);

    /* Print some information about the arrays */
    //     mexPrintf("Created shared data copy, and regular deep copy\n");
    mexPrintf("prhs[0] = %X, mxGetPr = %X, value = %lf\n",prhs[0],mxGetPr(prhs[0]),*mxGetPr(prhs[0]));
    mexPrintf("copy0   = %X, mxGetPr = %X, value = %lf\n",copy0,mxGetPr(copy0),*mxGetPr(copy0));
    mexPrintf("copy1   = %X, mxGetPr = %X, value = %lf\n",copy1,mxGetPr(copy1),*mxGetPr(copy1));
    mexPrintf("copy2   = %X, mxGetPr = %X, value = %lf\n",copy2,mxGetPr(copy2),*mxGetPr(copy2));

    /* TEST: Destroy the first copy */
    //mxDestroyArray(copy1);
    //copy1 = NULL;
    //mexPrintf("\nFreed copy1\n");
    /* RESULT: copy2 will still be valid */
    //mexPrintf("copy2 = %X, mxGetPr = %X, value = %lf\n",copy2,mxGetPr(copy2),*mxGetPr(copy2));

    if (nlhs>0) plhs[0] = mxCreateSharedDataCopy(prhs[0]);
    //if (nlhs>0) plhs[0] = const_cast<mxArray*>(prhs[0]);
}