How could I distinguish between NV21 and YV12 codification in imageReader camera API 2?

Question

I am developing custom camera API 2 app, and I notice that the capture format conversion is different on some devices when I use ImageReader callback.

For example in Nexus 4 doesn't work fine and in Nexus5X looks OK, here is the output.

I initialize the ImageReader in this form:

mImageReader = ImageReader.newInstance(320, 240, ImageFormat.YUV_420_888,2); 

And my callback is simple callback ImageReader Callback.

 mOnImageAvailableListener = new ImageReader.OnImageAvailableListener() {

    @Override
    public void onImageAvailable( ImageReader reader) {

       try {
             mBackgroundHandler.post(
                 new ImageController(reader.acquireNextImage())
             );
        }
        catch(Exception e)
        {
          //exception
        }
        }

};

And in the case of Nexus 4: I had this error.

D/qdgralloc: gralloc_lock_ycbcr: Invalid format passed: 0x32315659

When I try to write the raw file in both devices, I have these different images. So I understand that the Nexus 5X image has NV21 codification and the Nexus 4 has YV12 codification.

I found a specification of image format and I try to get the format in ImageReader. There are YV12 and NV21 options, but obviously, I get the YUV_420_888 format when I try to obtain the format.

 int test=mImageReader.getImageFormat();

So is there any way to get the camera input format (NV21 or YV12) to discriminate this codification types in the camera class? CameraCharacteristics maybe?

Thanks in advance.

Unai. PD: I use OpenGL for displayin RGB images, and I use Opencv to make the conversions to YUV_420_888.

Answer 1

YUV_420_888 is a wrapper that can host (among others) both NV21 and YV12 images. You must use the planes and strides to access individual colors:

ByteBuffer Y = image.getPlanes()[0];
ByteBuffer U = image.getPlanes()[1];
ByteBuffer V = image.getPlanes()[2];

If the underlying pixels are in NV21 format (as on Nexus 4), the pixelStride will be 2, and

int getU(image, col, row) {
    return getPixel(image.getPlanes()[1], col/2, row/2);
}

int getPixel(plane, col, row) {
    return plane.getBuffer().get(col*plane.getPixelStride() + row*plane.getRowStride());
}

We take half column and half row because this is how U and V (chroma) planes are stored in 420 image.

This code is for illustration, it is very inefficient, you probably want to access pixels at bulk, using get(byte[], int, int), or via a fragment shader, or via JNI function GetDirectBufferAddress in native code. What you cannot use, is method plane.array(), because the planes are guaranteed to be direct byte buffers.