What is the best camera calibration method?

Question

I understand the above question is vast and vague. But to put into context: I'm trying to determine the accuracy of pose and position estimation of a camera. I have spent weeks calibrating and trying different methods and different sized boards, lighting, distances etc.

The methods I have tried:

• OpenCV's built in calibration method
• Matlab Calibration Toolbox
• Matlab calibration App
• Manually adjusting values.

Description:

• Using about 20 images of various orientations and locations in front of the camera for each
• For a 9x6 checkerboard of sizes 25mm, 32mm and 50mm blocks
• At both resolutions of 1280x720 and 1920x1080
• At distances ranging between 500mm for the smaller board to 2000mm for the larger one

In all cases I have followed strict guidelines as per this link How to verify the correctness of calibration of a webcam?

In all combinations of the above factors, I get results that are a few millimeters off (+- 15mm) My intrinsics for 25mm blocks at 1280x720 for the above methods span as follows:

• OpenCV's built in calibration method > fx = 1269.4 fy = 1269.49 cx = 639.5 cy = 359.5
• Matlab Calibration Toolbox > fx = 1259.53 fy = 1260.76 cx = 661.3 cy = 306.5
• Matlab calibration App > fx = 1255.1 fy = 1254.8 cx = 652.6 cy = 340.7
• Manually adjusting values. > Various results, none stable with exception of cx = 639.5 cy = 359.5

This should not be the case. The position of the camera relative to the origin of the board should be accurate to an average of a millimeter or two, if not submillimeter accuracy when dealing with distances of under 1 meter. Unless I am mistaken?

My question is, what is an ideal, simple calibration method, for an HD webcam, with little distortion?

Answer 1

There are many possible sources of error.

First of all, while all three of the calibration implementations you have tried use essentially the same algorithm, there are enough differences that explain the discrepancies in the results.

The main difference is in the checkerboard corner detection. The Caltech Calibration Toolbox does not have automatic checkerboard detection, and uses a second optimization pass to refine the corners. Both OpenCV and the Camera Calibrator App do detect the checkerboard automatically, but the algorithm used in the Camera Calibrator App is far better. It is more robust, meaning that it is likely to detect a board when OpenCV does not, and its sub-pixel localization is more precise. My point is that in these three approaches you are calibrating using different data points. So it is not surprising that your results are different.

Once you have calibrated, what kind of reprojection errors are you getting? The Camera Calibrator App shows you a bar graph of the reprojection errors. You should look at it, and exclude the images that give you high errors. Ideally, you want your mean reprojection error to be less than half a pixel. The lower the better.

Now I have to ask you how you are measuring the distance from the camera to the checkerboard? The extrinsics you are getting from the calibration represent the transformation from the checkerboard's coordinate system into the camera's coordinate system, whose origin is inside the camera case, at its optical center. This is hard to measure accurately. A better way would be to place a flat object of a known size on the checkerboard and measure it using the camera. In fact, you can measure the distances between the detected checkerboard corners. Note that the detection accuracy is another source of error.

Another thing, please be sure not to save your calibration images as jpeg. The compression artifacts will affect the accuracy of the checkerboard corner detection. Use a lossless format like tiff or png.