How can I get Camera Calibration Data on iOS? aka AVCameraCalibrationData

Question

As I understand it, AVCameraCalibrationData is only available over AVCaptureDepthDataOutput. Is that correct?

AVCaptureDepthDataOutput on the other hand is only accessible with iPhone X front cam or iPhone Plus back cam, or am I mistaken?

What I am trying to do is to get the FOV of an AVCaptureVideoDataOutput SampleBuffer. Especially, it should match the selected preset (full HD, Photo etc.).

Answer 1

Background: A lot of these stack overflow responses are referencing intrinsic data when asked about camera calibration, including the accepted answer for this post, but calibration data typically includes intrinsic data, extrinsic data, lens distortion, etc. Its all listed out here in the iOS documentation. The author mentioned they were just looking for FOV, which is in the sample buffer, not in the camera calibration data. So ultimately, I think his question was answered. BUT if you found this question looking for actual camera calibration data, this will throw you off. And like the answer said, you can only get calibration data under specific conditions, which I outline more below.

Before I answer the rest, I would just say that the accepted answer here is great if you ARE looking for JUST the intrinsic matrix, that can be obtained much easier (i.e. not as stringent of an environment) than the rest of these values through the approach outlined above. If you are using this for computer vision, which is what I am using it for, that is sometimes all that is needed. But for really cool stuff, you'll want it all! So I will proceed to explain how to reach that:

I am going to assume you have the general camera app code in place. In that code, when a picture is taken, you are probably going to make a call to the photoOutput function that looks likes something like this:

func photoOutput(_ output: AVCapturePhotoOutput, didFinishProcessingPhoto photo: AVCapturePhoto, error: Error?) {...

The output parameter is going to have a value you can reference to see if camera calibration is supported called isCameraCalibrationDataDeliverySupported, so for example, to print that out, use something like this:

print("isCameraCalibrationDataDeliverySupported: \(output.isCameraCalibrationDataDeliverySupported)")

Note in the documentation I linked to, it is only supported in specific scenarios:

"This property's value can be true only when the isDualCameraDualPhotoDeliveryEnabled property is true. To enable camera calibration delivery, set the isCameraCalibrationDataDeliveryEnabled property in a photo settings object."

So that's important, pay attention to that to avoid unnecessary stress. Use the actual value to debug and make sure you have the proper environment enabled.

With all that in place, you should get the actual camera calibration data from:

photo.cameraCalibrationData

Just pull out of that object to get specific values you are looking for, such as:

photo.cameraCalibrationData?.extrinsicMatrix
photo.cameraCalibrationData?.intrinsicMatrix
photo.cameraCalibrationData?.lensDistortionCenter
etc.

Basically everything that is listed in the documentation that I linked to above.