Matlab engines within parallel loop

Question

I am having some trouble utilising multiple matlab engines from within a piece of parallelised code. I can successfully spawn multiple engines using engOpenSingleUse but am unable to communicate with more than one engine (ie. calls to engPutVariable fail).

As always, a minimal (VS) example:

#include "stdafx.h"
#include <engine.h>
#include <omp.h>

int _tmain(int argc, _TCHAR* argv[])
{
//First spawn the matlab engine sessions
Engine *m_Engines[2];
for (int i = 0; i < 2; i++)
{
    m_Engines[i] = engOpenSingleUse(NULL, NULL, NULL);
}

//Then spawn the worker threads...
#pragma omp parallel num_threads(2)
{   
    // Allocate an engine to each thread
    int thread_num = omp_get_thread_num();
    Engine *thisEngine = m_Engines[thread_num];

    #pragma omp for
    for (int i = 0; i < 10000; i++)
    {
        // Create an mxArray and stick some data in it
        mxArray* mM = NULL;
        mM = mxCreateDoubleMatrix(1, 1, mxREAL);
        double data[1] = { 1.0 };
        memcpy((void *)mxGetPr(mM), (void *)data, sizeof(data));

        // Send it across to matlab
        engPutVariable(thisEngine, "A", mM);
        // Run some algorithm
        engEvalString(thisEngine, "A=A+1;");
        // Retrieve result
        mM = engGetVariable(thisEngine, "A");

        // Get it out of the mxarray
        double A = *mxGetPr(mM);
    }
}

return 0;
}

Any ideas? Am using Matlab R2012b on Win x64.

Answer 1

As commented below your question, if you deliberatly call functions of a non-thread safe closed source library from multiple thread with no synchronization, then the behaviour is not predictable and you should avoid doing it. A possible idea is to span calls to MATLAB engine across multiple processes, but please consider that it may increase complexity (inter-process synchronization is required) and worsen performances.

By the way, C++11 and later users should program by using the new MATLAB Engine API for C++. Please note that engOpenSingleUse has not an equivalent function, at least not explicitly, in the new API (the closest replacement is matlab::engine::startMATLAB).