iOS; Unsupported Pixel Format

Question

My Unity app uses the Camera feed for multiple systems that the app uses; such as Vuforia, ARKit, and just general Camera feed input.

The issue I'm having is that each one of these requests a different Pixel Format, which seems to cause an issue for the ARKit. This requires YUV, and I don't know how to natively switch the camera pixel format back to this. So I get this error when I try and launch ARKit after the others;

2017-08-21 08:51:38.749452+0100 ar_app[399:105849] [Sensor] Unsupported pixel format: 875704438

2017-08-21 08:51:38.749876+0100 ar_app[399:105849] [Session] Session did fail with error: Error Domain=com.apple.arkit.error Code=104 "Unsupported capture session configuration."

UserInfo={ NSLocalizedRecoverySuggestion=Make sure that the correct device and format are being used for capture.,

NSLocalizedDescription=Unsupported capture session configuration.,

NSLocalizedFailureReason=Input device and/or format of the provided capture session are not supported for the given configuration. }

At least, thats what I think the issue is; that it's not getting the format in YUV. Any help would be much appreciated. . Thanks.

Answer 1

YUV is a color space, there are multiple formats.

ar_app[399:105849] [Sensor] Unsupported pixel format: 875704438

875704438 translates to 420v (NV12) or kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange:

kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange = '420v'

So you actually have a YUV pixel format, specifically the biplanar with video range.

The ARKit docs do mention the biplanar YUV format but not the type of range:

The pixel buffer’s contents are encoded in a biplanar YCbCr (also called YUV) data format

However the Unity ARKit Plugin does a check for kCVPixelFormatType_420YpCbCr8BiPlanarFullRange in didUpdateFrame:

 if (CVPixelBufferGetPlaneCount(pixelBuffer) < 2 || CVPixelBufferGetPixelFormatType(pixelBuffer) != kCVPixelFormatType_420YpCbCr8BiPlanarFullRange) {
        return;
    }

So in conclusion it might not like the fact that it's not full range, but I have no easy way to test it. Maybe you can try forcing full range if the camera and the other modules support it.