Usage of findEssentialMat function in OpenCV 3.0

Question

I'm currently working on a project to recover camera 6-DOF-Pose from two images by using SIFT/SURF. In old version of OpenCV, I use findFundamentalMat to find fundamental matrix, then further getting essential matrix with know camera intrinsic K and eventually get R and t by matrix decomposition. The result is very sensitive and unstable.

I saw some people have the same issue here OpenCV findFundamentalMat very unstable and sensitive

Some people suggest apply Nister's 5-point algorithm which has implemented in the latest version of OpenCV3.0.

I have read an examples from OpenCV documentation

In the example, it use focal = 1.0 and Point2d pp(0.0, 0.0). Is this the real focal length and principle point of the camera? what are the unit? in pixel? or in actual size? I am having trouble to understand these two parameters. I think these two parameters should be acquired from a calibration routine, right?

For my current camera (VGA mode), I use Matlab Camera Calibrator to get these two parameters, and these parameters are

Focal length (millimeters):   [  1104    1102]

Principal point (pixels):[  259      262]

So, if I want to use my camera parameters instead, should I need to directly fill in these values? or convert them to actual size, like millimeter?

Also, the translation result I get looks like a direction rather than actual size, is there any way I can get the actual size translation rather than a direction?

Any help is appreciated.

Answer 1

Focal Length

The focal length that you get from camera calibration is in pixels. It is actually the ratio of the "real" focal length (e.g. in mm) and the pixel size (also in mm). The world units cancel out, and you are left with pixels. Unfortunately, you cannot estimate both the focal length in world units and the pixel size, only their ratio.

Principal Point

The principal point is also in pixels. It is simply the point in the image where it intersects the optical axis. One caveat: the principal point you get from the Camera Calibrator in MATLAB uses 1-based pixel coordinates, where the center of the top-right pixel of the image is (1,1). OpenCV uses 0-based pixel coordinates. So if you want to use your camera parameters in OpenCV, you have to subtract 1 from the principal point.

Translation Vector

The translation vector you get from the essential matrix is a unit vector, because the essential matrix is only defined up to scale. In other words, you get a reconstruction where the unit is the distance between the cameras. If you need a metric reconstruction (in actual world units) you would either need to know the actual distance between the cameras, or you need to be able to detect an object of a known size in the scene. See this example.