InitUndistortRectifyMap and Remap

Question

I am currently writing an openCV program for a pair of stereo cameras. Camera calibration as well as Stereo Calibration are done.

The next step is to find a feature's position in space from the 2 images I get. That's why I have to Stereo Rectify the images and make my calculations afterwards.

The problem I am facing with initUndistortRectifyMap is the following:

-If I pass R1 or R2 calculated by stereoRectify() to initUndistortRectifyMap() I get black images after remapping.

-If I pass r (an empty matrix) to initUndistortRectifyMap() I get unrectified images after remapping. The images I get are a little distorted though.

I need to pass R1, and R2, to initUndistortRectifyMap() for rectifying the 2 cameras, otherwise when passing an empty matrix the stereo heads won't be rotated into the same plane.

Following is my code:

stereoRectify(intrinsic[0], distCoeffs[0], intrinsic[1], distCoeffs[1], imageSize, 
    R, T_Stereo, R1, R2, newP1, newP2, Q, CV_CALIB_ZERO_DISPARITY, -1, imageSize);

if (x_Cam.GetSerial()=="4002678487")
{
    initUndistortRectifyMap(intrinsic[0], distCoeffs[0], R1, newP1, imageSize,        
        CV_16SC2 , mapx1,  mapy1);
    remap(x_Image, imageRectified[0],mapx1, mapy1, INTER_LINEAR);

    return imageRectified[0];
}   

if (x_Cam.GetSerial()=="4002702131")
{
    //flip(in, in, -1);
    initUndistortRectifyMap(intrinsic[1], distCoeffs[1], R2, newP2, imageSize, 
        CV_16SC2 , mapx2,  mapy2);
    remap(x_Image, imageRectified[1],mapx2, mapy2, INTER_LINEAR, BORDER_CONSTANT, 0);

    return imageRectified[1];
}

I checked all the matrix values going in to stereoRectify() and they are correct. The rotation matrices R1, and R2 seem to be correct as well. I am just getting black images as output.

I tried passing rubbish values into InitUndistortRectifyMap() for R1 and R2 (R1*R2 for example) to see the effect simply, and I did get weird results but not black images.

Answer 1

Well, I kind of went around the problem and thought I'd share the solution incase anyone can use it.

InitUndistortRectifyMap did output blank images when using the Rotation Matrices R1, and R2 produced by StereoRectify.

Therefore, I tried to use StereoRectifyUncalibrated, which produces 2 Homography matrices H1, and H2, and the I calculated the rotations according to OpenCV documentation:

R1 = inv(CamMatrix1)*H1*CamMatrix1 R2 = inv(CamMatrix2)*H2*CamMatrix2

I passed the new R1, and R2 to InitUndistortRectifyMap and remapped and the results are satisfying.