OpenCV element-wise matrix multiplication

Question

OpenCV docs say A.mul(B) is per-element multiplication. Yet the following code produces the following output, and then gives this error:

OpenCV Error: Sizes of input arguments do not match

.

cout << laplacian_pyramids[i][numLevels - 1 - l].rows << endl;
cout << gaussian_weight_pyramids[i][l].rows << endl;
cout << laplacian_pyramids[i][numLevels - 1 - l].cols << endl;
cout << gaussian_weight_pyramids[i][l].cols << endl;

Gives:

339
339
571
571

Then:

Mat prod = gaussian_weight_pyramids[i][l].mul(laplacian_pyramids[i][numLevels - 1 - l]);

gives the error. I tried Mat::multiply to a similar effect.

Answer 1

I would recommend converting single channel to three channels:

    Mat A = Mat::zeros(100, 200, CV_32FC1);
    Mat B = Mat::zeros(100, 200, CV_32FC3);

    // Mat C = A.mul(B); // Sizes of input arguments do not match

    Mat Afc3;
    Mat t[] = {A, A, A};
    merge(t, 3, Afc3);

    Mat C = Afc3.mul(B); // now Afc3 has 3 channels ans it is type of 32_FC3 
                         // we can multiply each elem in B by the same coef from A

But if B it is a CV_8UC3 type, it does not work because opencv would not allow to multiply Mats which have different types of pixels. In that case, convert 8UC3 to 32FC3 remebering to scale each pixel by 1/255.0 beacuse each pixel in 32FC3 has a value between 0.0 and 1.0 (and of course each pixel in 8UC3 has a value between 0 and 255).

    Mat A = Mat::zeros(100, 200, CV_32FC1);
    Mat B = Mat::zeros(100, 200, CV_8UC3);

    // Mat C = A.mul(B);

    Mat Afc3, Bfc3;
    Mat t[] = {A, A, A};
    merge(t, 3, Afc3);

    B.convertTo(Bfc3, CV_32FC3, 1/255.0);

    Mat C = Afc3.mul(Bfc3);

Answer 2

There can be 2 reasons for such error: different number of channels or different type of data (for example if first matrix contain unsigned char and second matrix contain unsigned short). Of course there can be both reasons. In general there 3 types of solutions for problems like the one you encountered:

1) Write your own 'for' loop that will do the operation you need. You won't benefit from optimizations that might be present in OpenCV functions but other solutions will have their own overheads. You can see this tutorial about how to access pixels in efficient way.

2) Use functions like 'merge' or 'convertTo' in order to create input of same type and number of channels. See answer posted by @marol for code example. In this solution the main overhead is copy of data. That means extra time and space. This is reasonable solution if you are going to perform multiple operations with both images. But if all you need is simple multiplication it won't be very effective.

3) Use workarounds. For example if your matrices have same type but differ in number of channels you can use reshape function:

// two matrices of same size but different number of channels
Mat laplac(100, 200, CV_32FC3);
Mat gauss(100, 200, CV_32FC1);

// turn them into single channel matrices. they have NxM rows and 1 or 3 columns.
// note that there no copy of data. any change in them will affect original matrices
Mat laplac2 = laplac.reshape( 1, laplac.rows*laplac.cols );
Mat gauss2 = gauss.reshape( 1, gauss.rows*gauss.cols ;

// perform multiplication
laplac2.col(0) = laplac2.col(0).mul(gauss2);
laplac2.col(1) = laplac2.col(1).mul(gauss2);
laplac2.col(2) = laplac2.col(2).mul(gauss2);

This way you are using only OpenCV build-in functions without copy overhead. But I doubt that this will be any faster than solution-1, because solution-1 is more efficient in terms of memory access.

In any case you won't have nice and clean operation that takes exactly one line :(