matlab matrix operation speed

Question

I've been asked to make some MATLAB code run faster, and have run into something that seems strange to me.

In one of the functions there's a loop where we multiply a 3x1 vector (let's call it x) - a 3x3 matrix (let's call it A) - and the transpose of x, yielding a scalar. The code has the whole set of element-by-element multiplications and additions, and is pretty cumbersome:

val = x(1)*A(1,1)*x(1) + x(1)*A(1,2)*x(2) + x(1)*A(1,3)*x(3) + ...
      x(2)*A(2,1)*x(1) + x(2)*A(2,2)*x(2) + x(2)*A(2,3)*x(3) + ... 
      x(3)*A(3,1)*x(1) + x(3)*A(3,2)*x(2) + x(3)*A(3,3)*x(3);

I figured I'd just replace it all by:

val = x*A*x';

To my surprise, it ran significantly slower (as in 4-5 times slower). Is it just that the vector and matrix are so small that MATLAB's optimizations don't apply?

Answer 1

EDIT: I improved the tests to give more accurate times. I also optimized the unrolled version which is now much better than what I initially had, still matrix multiplication is way faster as you increase the size.

EDIT2: To make sure that the JIT compiler is working on the unrolled functions, I modified the code to write the generated functions as M-files. Also the comparison can now be seen as fair as both methods get evaluated by passing TIMEIT the function handle: timeit(@myfunc)

---

I am not convinced that your approach is faster than matrix multiplication for reasonable sizes. So lets compare the two methods.

I am using the Symbolic Math Toolbox to help me get the "unrolled" form of the equation of x'*A*x (try multiplying by hand a 20x20 matrix and a 20x1 vector!):

function f = buildUnrolledFunction(N)
    % avoid regenerating files, CCODE below can be really slow!
    fname = sprintf('f%d',N);
    if exist([fname '.m'], 'file')
        f = str2func(fname);
        return
    end

    % construct symbolic vector/matrix of the specified size
    x = sym('x', [N 1]);
    A = sym('A', [N N]);

    % work out the expanded form of the matrix-multiplication
    % and convert it to a string
    s = ccode(expand(x.'*A*x));    % instead of char(.) to avoid x^2

    % a bit of RegExp to fix the notation of the variable names
    % also convert indexing into linear indices: A(3,3) into A(9)
    s = regexprep(regexprep(s, '^.*=\s+', ''), ';$', '');
    s = regexprep(regexprep(s, 'x(\d+)', 'x($1)'), 'A(\d+)_(\d+)', ...
        'A(${ int2str(sub2ind([N N],str2num($1),str2num($2))) })');

    % build an M-function from the string, and write it to file
    fid = fopen([fname '.m'], 'wt');
    fprintf(fid, 'function v = %s(A,x)\nv = %s;\nend\n', fname, s);
    fclose(fid);

    % rehash path and return a function handle
    rehash
    clear(fname)
    f = str2func(fname);
end

I tried to optimize the generated function by avoid exponentiation (we prefer x*x to x^2). I also converted the subscripts into linear indices (A(9) instead of A(3,3)). Therefore for n=3 we get the same equation you had:

>> s
s =
A(1)*(x(1)*x(1)) + A(5)*(x(2)*x(2)) + A(9)*(x(3)*x(3)) + 
A(4)*x(1)*x(2) + A(7)*x(1)*x(3) + A(2)*x(1)*x(2) + 
A(8)*x(2)*x(3) + A(3)*x(1)*x(3) + A(6)*x(2)*x(3)

Given the above method to construct M-functions, we now evaluate it for various sizes and compare it against the matrix-multiplication form (I put it in a separate function to account for function calling overhead). I am using the TIMEIT function instead of tic/toc to get more accurate timings. Also to have a fair comparison, each method is implemented as an M-file function that gets passed all of the needed variables as input arguments.

function results = testMatrixMultVsUnrolled()
    % vector/matrix size
    N_vec = 2:50;
    results = zeros(numel(N_vec),3);
    for ii = 1:numel(N_vec);
        % some random data
        N = N_vec(ii);
        x = rand(N,1); A = rand(N,N);

        % matrix multiplication
        f = @matMult;
        results(ii,1) = timeit(@() feval(f, A,x));

        % unrolled equation
        f = buildUnrolledFunction(N);
        results(ii,2) = timeit(@() feval(f, A,x));

        % check result
        results(ii,3) = norm(matMult(A,x) - f(A,x));
    end

    % display results
    fprintf('N = %2d: mtimes = %.6f ms, unroll = %.6f ms [error = %g]\n', ...
        [N_vec(:) results(:,1:2)*1e3 results(:,3)]')
    plot(N_vec, results(:,1:2)*1e3, 'LineWidth',2)
    xlabel('size (N)'), ylabel('timing [msec]'), grid on
    legend({'mtimes','unrolled'})
    title('Matrix multiplication: $$x^\mathsf{T}Ax$$', ...
        'Interpreter','latex', 'FontSize',14)
end

function v = matMult(A,x)
    v = x.' * A * x;
end

The results:

N =  2: mtimes = 0.008816 ms, unroll = 0.006793 ms [error = 0]
N =  3: mtimes = 0.008957 ms, unroll = 0.007554 ms [error = 0]
N =  4: mtimes = 0.009025 ms, unroll = 0.008261 ms [error = 4.44089e-16]
N =  5: mtimes = 0.009075 ms, unroll = 0.008658 ms [error = 0]
N =  6: mtimes = 0.009003 ms, unroll = 0.008689 ms [error = 8.88178e-16]
N =  7: mtimes = 0.009234 ms, unroll = 0.009087 ms [error = 1.77636e-15]
N =  8: mtimes = 0.008575 ms, unroll = 0.009744 ms [error = 8.88178e-16]
N =  9: mtimes = 0.008601 ms, unroll = 0.011948 ms [error = 0]
N = 10: mtimes = 0.009077 ms, unroll = 0.014052 ms [error = 0]
N = 11: mtimes = 0.009339 ms, unroll = 0.015358 ms [error = 3.55271e-15]
N = 12: mtimes = 0.009271 ms, unroll = 0.018494 ms [error = 3.55271e-15]
N = 13: mtimes = 0.009166 ms, unroll = 0.020238 ms [error = 0]
N = 14: mtimes = 0.009204 ms, unroll = 0.023326 ms [error = 7.10543e-15]
N = 15: mtimes = 0.009396 ms, unroll = 0.024767 ms [error = 3.55271e-15]
N = 16: mtimes = 0.009193 ms, unroll = 0.027294 ms [error = 2.4869e-14]
N = 17: mtimes = 0.009182 ms, unroll = 0.029698 ms [error = 2.13163e-14]
N = 18: mtimes = 0.009330 ms, unroll = 0.033295 ms [error = 7.10543e-15]
N = 19: mtimes = 0.009411 ms, unroll = 0.152308 ms [error = 7.10543e-15]
N = 20: mtimes = 0.009366 ms, unroll = 0.167336 ms [error = 7.10543e-15]
N = 21: mtimes = 0.009335 ms, unroll = 0.183371 ms [error = 0]
N = 22: mtimes = 0.009349 ms, unroll = 0.200859 ms [error = 7.10543e-14]
N = 23: mtimes = 0.009411 ms, unroll = 0.218477 ms [error = 8.52651e-14]
N = 24: mtimes = 0.009307 ms, unroll = 0.235668 ms [error = 4.26326e-14]
N = 25: mtimes = 0.009425 ms, unroll = 0.256491 ms [error = 1.13687e-13]
N = 26: mtimes = 0.009392 ms, unroll = 0.274879 ms [error = 7.10543e-15]
N = 27: mtimes = 0.009515 ms, unroll = 0.296795 ms [error = 2.84217e-14]
N = 28: mtimes = 0.009567 ms, unroll = 0.319032 ms [error = 5.68434e-14]
N = 29: mtimes = 0.009548 ms, unroll = 0.339517 ms [error = 3.12639e-13]
N = 30: mtimes = 0.009617 ms, unroll = 0.361897 ms [error = 1.7053e-13]
N = 31: mtimes = 0.009672 ms, unroll = 0.387270 ms [error = 0]
N = 32: mtimes = 0.009629 ms, unroll = 0.410932 ms [error = 1.42109e-13]
N = 33: mtimes = 0.009605 ms, unroll = 0.434452 ms [error = 1.42109e-13]
N = 34: mtimes = 0.009534 ms, unroll = 0.462961 ms [error = 0]
N = 35: mtimes = 0.009696 ms, unroll = 0.489474 ms [error = 5.68434e-14]
N = 36: mtimes = 0.009691 ms, unroll = 0.512198 ms [error = 8.52651e-14]
N = 37: mtimes = 0.009671 ms, unroll = 0.544485 ms [error = 5.68434e-14]
N = 38: mtimes = 0.009710 ms, unroll = 0.573564 ms [error = 8.52651e-14]
N = 39: mtimes = 0.009946 ms, unroll = 0.604567 ms [error = 3.41061e-13]
N = 40: mtimes = 0.009735 ms, unroll = 0.636640 ms [error = 3.12639e-13]
N = 41: mtimes = 0.009858 ms, unroll = 0.665719 ms [error = 5.40012e-13]
N = 42: mtimes = 0.009876 ms, unroll = 0.697364 ms [error = 0]
N = 43: mtimes = 0.009956 ms, unroll = 0.730506 ms [error = 2.55795e-13]
N = 44: mtimes = 0.009897 ms, unroll = 0.765358 ms [error = 4.26326e-13]
N = 45: mtimes = 0.009991 ms, unroll = 0.800424 ms [error = 0]
N = 46: mtimes = 0.009956 ms, unroll = 0.829717 ms [error = 2.27374e-13]
N = 47: mtimes = 0.010210 ms, unroll = 0.865424 ms [error = 2.84217e-13]
N = 48: mtimes = 0.010022 ms, unroll = 0.907974 ms [error = 3.97904e-13]
N = 49: mtimes = 0.010098 ms, unroll = 0.944536 ms [error = 5.68434e-13]
N = 50: mtimes = 0.010153 ms, unroll = 0.984486 ms [error = 4.54747e-13]

At small sizes the two methods perform somewhat similarly. Although for N<7 the expanded version beats mtimes, but the difference is hardly significant. Once we move past tiny sizes, matrix multiplication is orders of magnitude faster.

This is not really surprising; with only N=20 the formula is scary long and involves adding 400 terms. As MATLAB language is interpreted, I doubt this is very efficient..

---

Now I agree that there is an overhead for calling an external function vs. directly embedding the code in-line, but how practical is such an approach. Even for a small size as N=20, the generated line is over 7000 characters! I also noticed the MATLAB editor becoming sluggish on account of the long lines :)

Besides, the advantage quickly disappears after around N>10. I compared the embedded-code/explicitly-written vs. matrix-multiplication, similar to what @DennisJaheruddin suggested. The results:

N=3:
  Elapsed time is 0.062295 seconds.    % unroll
  Elapsed time is 1.117962 seconds.    % mtimes

N=12:
  Elapsed time is 1.024837 seconds.    % unroll
  Elapsed time is 1.126147 seconds.    % mtimes

N=19:
  Elapsed time is 140.915138 seconds.  % unroll
  Elapsed time is 1.305382 seconds.    % mtimes

... and it only gets worse for the unrolled version. Like I said before, MATLAB is interpreted so the cost of parsing the code starts to show at such huge files.

The way I see it, after doing a million iterations we only gained 1 second at best, which I think does not justify all the trouble and hacks, over using the much more readable and succinct v=x'*A*x. So perhaps there are other places in the code one can improve, rather than focusing on an already optimized operation such as matrix multiplication.

Matrix multiplication in MATLAB is seriously fast (this is what MATLAB is best at!). It really shines once you reach large enough data (as multithreading kicks in):

>> N=5000; x=rand(N,1); A=rand(N,N);
>> tic, for i=1e4, v=x.'*A*x; end, toc
Elapsed time is 0.021959 seconds.