Is is possible to calibrate fish eye camera using standard openCV?

Question

I try to calibrate camera with ultra wide angle lens > 180 degrees. I use OpenCV 3.2 and camera calibration code from opencv/samples/cpp/tutorial_code/calib3d/camera_calibration, as a result I get image similar to this one from this post. I found numerous movies on youtube.com on which people show how they calibrate fisheye cameras, for example this video.

I know that calibration of ultra wide lenses cameras could be tricky. I also found that there are another then standard model for fisheye camera calibrations in openCV contrib

Is it possible to use camera calibration model from standard OpenCV to calibrate such a wide angle camera or should I use another model e.g., from openCV contrib?

Answer 1

I was working on the same problem a few weeks ago, what I found was that the standard model did not work with my fisheye images ( >180 FOV). Having said that, I did find a pull request on Github which addresses just this issue - https://github.com/opencv/opencv/pull/6801.

I eventually gave up on the standard model and just used the Omnidirectional model - http://docs.opencv.org/trunk/dd/d12/tutorial_omnidir_calib_main.html which worked like a charm.

Just as a side note, whilst I was researching the topic of distortion correction I came across a common misconception that fisheye images resulting from fisheye lenses are "distorted", indeed they might be distorted but the resulting images are simply a projection. A fisheye like other projections is one of many ways of mapping a 3D world onto a 2D plane. It doesn't help that majority of the related literature refer to it as "distortion". Fisheye lenses are deliberately designed to produce images with bent lines, when these lines would have been straight with standard (i.e. rectilinear) lenses. The word "distortion" is reserved solely for geometric aberrations that negatively affect standard (i.e. rectilinear) lenses.

In my case, even though I was looking into distortion correction I didn't end up correcting for any distortion, I just simply did a fisheye to perspective projection - you can find more on this on here http://paulbourke.net/dome/fish2/

Also, in regards to capturing checkerboard images follow these recommendations - How to verify the correctness of calibration of a webcam?

Answer 2

I solved this problem for a 180 degrees FOV fisheye camera via cv::fisheye module from Open CV 3.4.0. (C++, MS Windows).

I used cv::fisheye::calibrate to make K and D (camera matrix and radial distortion coeffitients matrix). Then I used cv::fisheye::initUndistortRectifyMap to produce maps for X and Y coordinates. And finally I used cv::remap to undistort image from fisheye camera via maps from initUndistortRectifyMap. This solution has some limitations (OpenCV dewarps only a central part of fisheye image. Edges are moved outside) You'll get 140-150 FOV as a result. Detailed explanation of this limitation I placed HERE. If 140-150 degrees FOV is suitable for you, you can use this module without any modifications