How do we do rectilinear image conversion with swift and iOS 11+

Question

How do we use the function apple provides (below) to perform rectilinear conversion?

Apple provides a reference implementation in 'AVCameraCalibrationData.h' on how to correct images for lens distortion. Ie going from images taken with a wide-angle or telephoto lens to the rectilinear 'real world' image. A pictoral representation is here:

To create a rectilinear image we must begin with an empty destination buffer and iterate through it row by row, calling the sample implementation below for each point in the output image, passing the lensDistortionLookupTable to find the corresponding value in the distorted image, and write it to your output buffer.

func lensDistortionPoint(for point: CGPoint, lookupTable: Data, distortionOpticalCenter opticalCenter: CGPoint, imageSize: CGSize) -> CGPoint {
    // The lookup table holds the relative radial magnification for n linearly spaced radii.
    // The first position corresponds to radius = 0
    // The last position corresponds to the largest radius found in the image.

    // Determine the maximum radius.
    let delta_ocx_max = Float(max(opticalCenter.x, imageSize.width  - opticalCenter.x))
    let delta_ocy_max = Float(max(opticalCenter.y, imageSize.height - opticalCenter.y))
    let r_max = sqrt(delta_ocx_max * delta_ocx_max + delta_ocy_max * delta_ocy_max)

    // Determine the vector from the optical center to the given point.
    let v_point_x = Float(point.x - opticalCenter.x)
    let v_point_y = Float(point.y - opticalCenter.y)

    // Determine the radius of the given point.
    let r_point = sqrt(v_point_x * v_point_x + v_point_y * v_point_y)

    // Look up the relative radial magnification to apply in the provided lookup table
    let magnification: Float = lookupTable.withUnsafeBytes { (lookupTableValues: UnsafePointer<Float>) in
        let lookupTableCount = lookupTable.count / MemoryLayout<Float>.size

        if r_point < r_max {
            // Linear interpolation
            let val   = r_point * Float(lookupTableCount - 1) / r_max
            let idx   = Int(val)
            let frac  = val - Float(idx)

            let mag_1 = lookupTableValues[idx]
            let mag_2 = lookupTableValues[idx + 1]

            return (1.0 - frac) * mag_1 + frac * mag_2
        } else {
            return lookupTableValues[lookupTableCount - 1]
        }
    }

    // Apply radial magnification
    let new_v_point_x = v_point_x + magnification * v_point_x
    let new_v_point_y = v_point_y + magnification * v_point_y

    // Construct output
    return CGPoint(x: opticalCenter.x + CGFloat(new_v_point_x), y: opticalCenter.y + CGFloat(new_v_point_y))
}

Additionally apple states: "point", "opticalCenter", and "imageSize" parameters below must be in the same coordinate system.

With that in mind, what values do we pass for opticalCenter and imageSize and why? What exactly is the "applying radial magnification" doing?

Answer 1

1. The opticalCenter is actually named distortionOpticalCenter. So you can provide lensDistortionCenter from AVCameraCalibrationData.

2. Image size is a height and width of image you want to rectilinear.

3. "Applying radial magnification". It changes the coordinates of given point to the point where it will be with ideal lens without distortion.

4. "How do we use the function...". We should create an empty buffer with same size as the distorted image. For each pixel of empty buffer we should apply the lensDistortionPointForPoint function. And take a pixel with corrected coordinates from distorted image to empty buffer. After fill all buffer space you should get an undistorted image.