Calculate and use inverse of OpenCV distortion parameters

Question

So basically I want to find out what the invert distortion parameters would be for a calibration I do as showed here.

NOTE: I know that we can perform undistortion and then use remapping to do what I have done below, but my goal is to be able to find out the actual inverted distortion parameters and use them to distort other images, not to just be able to revert what cv2.undistort() does

---

Overview:

I have tried passing in the negation of the distortion parameters:

# _, mat, distortion, _, _ = cv2.calibrateCamera(...)
# undistorted_image = cv2.undistort(with distortion) 
# redistorted_image = cv2.undistort(with np.negative(distortion))

In theory, I was thinking that if the redistorted_image is similar to the original image, then the np.negative(distortion) parameters are what I am looking for, but it turned out to be false.

---

Actual method I use:

def save_undistorted_images(image, matrix, distortion):
    test_image_su = cv.imread(image)
    height, width = test_image_su.shape[:2]
    new_camera_mtx, roi = cv.getOptimalNewCameraMatrix(matrix, distortion, (width, height), 1, (width, height))

    distortion_u = np.negative(distortion)

    # unsure if this line helps
    new_camera_mtx_inv, roi = cv.getOptimalNewCameraMatrix(matrix, distortion_u, (width, height), 1, (width, height))

    # undistort the image
    undistorted_image = cv.undistort(test_image_su, matrix, distortion, None, new_camera_mtx)
    cv.imwrite('undistorted_frame.png', undistorted_image)

    # redistort trying to get something like original image (image_test_su) 
    distorted_image = cv.undistort(undistorted_image, matrix, distortion_u, None, new_camera_mtx_inv)
    cv.imwrite('redistorted_frame.png', distorted_image)

The results:

(left a: original) (right b: undistorted)

(left c: distorted using np.negative(distortion)) (right d: undistorted image redistorted using np.negative(distortion)))

The image d here is basically c performed on b, which I expected would be similar to a

Why is b here overpowering the effect of c?

---

Other way of calculating inverse that I tried:

The following is my python implementation of this paper

    distortion_u = distortion

    k1 = distortion_u[0][0]
    k2 = distortion_u[0][1]
    k3 = distortion_u[0][4]

    b1 = -1 * k1
    b2 = 3 * (k1 * k1) - k2
    b3 = (8 * k1 * k2) + (-1 * (12 * (k1 * k1 * k1))) - k3
    
    # radial:
    distortion_u[0][0] = b1
    distortion_u[0][1] = b2
    distortion_u[0][4] = b3
   
    # tangential: 
    #distortion_u[0][2] = -1 * distortion_u[0][2]
    #distortion_u[0][3] = -1 * distortion_u[0][3]

The results of applying distortion on undistorted image using above distortion parameters is also not good, looks really similar to results above.

---

So, this brings us to:

Why is the effect of normal distortion always overpowering np.negative(distortion) or anything else?

Does all distortion work this way? (negative values does not equal to opposite effect)

How to get the actually opposite distortion parameters?

Answer 1

Afraid you are doing it wrong. The opencv distortion model you have calibrated computes undistorted and normalized image coordinates from distorted ones. It is a nonlinear model, so inverting it involves solving a system of nonlinear (polynomial) equations.

A closed form (parametric) solution exists AFAIK only for the case of single-parameter pure radial distortion, i.e. when the only nonzero distortion parameter is k1, the coefficient of r^2. In this case the model inversion equation reduces to a cubic equation in r, and you can then express the inverse model using Cardano's formula for the solution of the cubic.

In all other cases one inverts the model numerically, using various algorithms for solving the nonlinear system of equations. OpenCV uses an iterative "false-position" method.

Since you want to use the inverse model to un-distort a set of images (which is the normal use case), you should use initUndistortRectifyMap to calculate the undistortion solution for the image once and for all, and then pass it for every image to remap to actually undistort the images.

If you really need a parametric form for the inverse model, my advice would be to look into approximating the maps returned by initUndistortRectifyMap with a pair of higher order polynomials, or thin-plate splines.