OpenCV CV::Mat and Eigen::Matrix

Question

Is there a reversible way to convert an OpenCV cv::Mat object to an Eigen::Matrix?

e.g., Some way of doing:

cv::Mat cvMat;
Eigen::Matrix eigMat;
camera->retrieve(cvMat);

// magic to convert cvMat to eigMat
// work on eigMat
// convert eigMat back to cvMat

imshow("Image", cvMat);

I've tried using cv2eigen and eigen2cv, but the resulting cvMat is completely mangled and I'm not exactly sure why. The dimensions are correct, but the graphics are totally trashed, so possibly a bytes-per-pixel or datasize issue?

Answer 1

You can also use

void eigen2cv(const Eigen::Matrix<_Tp, _rows, _cols, _options, _maxRows, _maxCols>& src, Mat& dst)

and

void cv2eigen(const Mat& src, Eigen::Matrix<_Tp, _rows, _cols, _options, _maxRows, _maxCols>& dst)

from #include <opencv2/core/eigen.hpp>.

Answer 2

You should consider using Eigen::Map to wrap OpenCV matrices in order to be used directly by the Eigen SDK. This allows you to apply almost all functionalities implemented in Eigen on matrix allocated by OpenCV

In particular you simply instantiate an Eigen::Map providing the pointer to the cv::Mat buffer:

//allocate memory for a 4x4 float matrix
cv::Mat cvT(4,4,CV_32FC1); 

//directly use the buffer allocated by OpenCV
Eigen::Map<Matrix4f> eigenT( cvT.data() ); 

for more information on Eigen::Map take a look at Eigen Tutorial: Map Class

Answer 3

You can map arbitrary matrices between Eigen and OpenCV (without copying data).

You have to be aware of two things though:

• Eigen defaults to column-major storage, OpenCV stores row-major. Therefore, use the Eigen::RowMajor flag when mapping OpenCV data.

• The OpenCV matrix has to be continuous (i.e. ocvMatrix.isContinuous() needs to be true). This is the case if you allocate the storage for the matrix in one go at the creation of the matrix (e.g. as in my example below, or if the matrix is the result of a operation like Mat W = A.inv();)

Example:

Mat A(20, 20, CV_32FC1);
cv::randn(A, 0.0f, 1.0f); // random data

// Map the OpenCV matrix with Eigen:
Eigen::Map<Eigen::Matrix<float, Eigen::Dynamic, Eigen::Dynamic, Eigen::RowMajor>> A_Eigen(A.ptr<float>(), A.rows, A.cols);

// Do something with it in Eigen, create e.g. a new Eigen matrix:
Eigen::Matrix<float, Eigen::Dynamic, Eigen::Dynamic, Eigen::RowMajor> B = A_Eigen.inverse();

// create an OpenCV Mat header for the Eigen data:
Mat B_OpenCV(B.rows(), B.cols(), CV_32FC1, B.data());

For multi-channel matrices (e.g. images), you can use 'Stride' exactly as Pierluigi suggested in his comment!