Understanding MATLAB on multiple cores, multiple processors and MPI

Question

I have several closely related questions about how how MATLAB takes advantage of parallel hardware. They are short, so I thought it would be best to put them in the same post:

• Does MATLAB leverage/benefit from multiple cores when not using the Parallel Computing Toolbox?
• Does MATLAB leverage/benefit from multiple processors when not using the PCT?
• Does MATLAB use MPI when not using the PCT?
• Does MATLAB use MPI when using the PCT?

Answer 1

Does MATLAB leverage/benefit from multiple cores when not using the Parallel Computing Toolbox?

Yes. Since R2007a, more and more built-in functions have been re-written to be multi-threaded (though multi-threading will only kick in if it's beneficial).

Element Wise Functions and Expressions:
------------------------------------------------------------------------------------------------
Functions that speed up for double arrays > 20k elements 

1) Trigonometric: ACOS(x), ACOSH(x), ASIN(x), ASINH(x), ATAN(x), ATAND(x), ATANH(x), COS(x), COSH(x), SIN(x), SINH(x), TAN(x), TANH(x)

2) Exponential: EXP(x), POW2(x), SQRT(x)

3) Operators: x.^y
For Example: 3*x.^3+2*x.^2+4*x +6, sqrt(tan(x).*sin(x).*3+8);

Functions that speed up for double arrays > 200k elements 

4) Trigonometric: HYPOT(x,y), TAND(x)

5) Complex: ABS(x)

6) Rounding and remainder: UNWRAP(x), CEIL(x), FIX(x), FLOOR(x), MOD(x,N), ROUND(x)

7) Basic and array operations: LOGICAL(X), ISINF(X), ISNAN(X), INT8(X), INT16(X), INT32(X)

Linear Algebra Functions: 
------------------------------------------------------------------------------------------------
Functions that speed up for double arrays > 40k elements (200 square) 


1)Operators: X*Y (Matrix Multiply), X^N (Matrix Power)

2)Reduction Operations : MAX and MIN (Three Input), PROD, SUM

3) Matrix Analysis: DET(X), RCOND(X), HESS(X), EXPM(X)

4) Linear Equations: INV(X), LSCOV(X,x), LINSOLVE(X,Y), A\b (backslash)

5) Matrix Factorizations: LU(X), QR(X) for sparse matrix inputs

6) Other Operations: FFT and IFFT of multiple columns of data, FFTN, IFFTN, SORT, BSXFUN, GAMMA, GAMMALN, ERF,ERFC,ERFCX,ERFINV,ERFCINV, FILTER

For code implemented as .m file, multiple cores won't help, though.

Multi-threaded mex-files will benefit as well, of course.

Does MATLAB use MPI when not using the PCT?

Not to my knowledge.

Does MATLAB use MPI when using the PCT?

Yes, when you run it on a cluster (though you can use other schedulers as well). To do this, you need a license for the Matlab Distributed Computing Server license. I don't know what architecture the local scheduler uses (the one you use when you run parallel jobs on a local machine); given that MPI functions are part of the PCT suggests that they may use it for at least part of the functionality. EDIT: See @Edric's answer for more details

Answer 2

To clarify and expand on a couple of points from @Jonas' detailed answer:

• PCT uses a build of MPICH2 (this is not shipped with base MATLAB).
• MPI functions are available under the local scheduler - in this case, the build of MPICH2 can take advantage of shared memory for communication.
• The labSend/labReceive family of functions present a wrapper around MPI_Send/MPI_Recv etc.

Answer 3

When not using the PCT, MatLab issues only one command at once (single-threaded).

However, if you have a multi-threaded BLAS, you could still benefit from extra cores (and it doesn't particularly matter whether they're all in a single processor or not).

MEX files can also be written with multiple threads, in which case you will use multiple cores even without the PCT. If you have performance problems, rewriting some of the hotspots as MEX is often a big win.