How is the reprojection error calculated in Matlab's triangulate function? Sadly, the documentation gives no mathematical formula

Question

How is the reprojection error calculated in Matlab's triangulate function?

Sadly, the documentation gives no mathematical formula.

It only says: The vector contains the average reprojection error for each M world point.

What is the procedure/Matlab uses when calculating this error?

I searched SOF but found nothing on this IMHO important question.

UPDATE: How can they use this error to filter out bad matches here : http://se.mathworks.com/help/vision/examples/sparse-3-d-reconstruction-from-two-views.html

Answer 1

AFAIK The reprojection error is calculated always in the same way (in the field of computer vision in general).

The reprojection is (as the name says) the error between the reprojected point in the camera and the original point.

So from 2 (or more) points in the camera you triangulate and get 3D points in the world system. Due to errors in the calibration of the cameras, the point will not be 100% accurate. What you do is take the result 3D point (P) and with the camera calibration parameters project it in the cameras again, obtaining new points (\hat{p}) near the original ones (p).

Then you calculate the euclidean distance between the original point and the "reprojected" one.

In case you want to know a bit more about the method used by Matlab, I'll enhance the bibliography they use giving you also the page number:

Multiple View Geometry in Computer Vision by Richard Hartley and Andrew Zisserman (p312). Cambridge University Press, 2003.

But basically it is a least squares minimization, that has no geometrical interpretation.

Answer 2

You can find an explanation of reprojection errors in the context of camera calibration in the Camera Calibrator tutorial:

The reprojection errors returned by the triangulate function are essentially the same concept.

The way to use the reprojection errors to discard bad matches is shown in this example:

[points3D, reprojErrors] = triangulate(matchedPoints1, matchedPoints2, ...
    cameraMatrix1, cameraMatrix2);

% Eliminate noisy points
validIdx = reprojErrors < 1;
points3D = points3D(validIdx, :);

This code excludes all 3D points for which the reprojection error was more than a pixel. You can also use validIdx to eliminate the corresponding 2D matches.