opencv C++ create Mat object from android NV21 image data buffer

Question

I have implemented an android application that starts the camera and send all the preview buffer down to native components using JNI interface. Since the preview data is in NV21 image format, I need to create a cv::Mat instance from it. I searched for it and found the below solution:

cv::Mat _yuv(height, width, CV_8UC1, (uchar *) imagebuffer);

where imagebuffer is jbyte*

However, don't get the expected image in the output image. It's all filled with some random lines etc. Does anyone know how exactly I can do it?

Answer 1

You need to convert YUV image to RGBA image.

cv::Mat _yuv(height+height/2, width, CV_8UC1, (uchar *)imagebuffer);
cv::cvtColor(_yuv, _yuv, CV_YUV2RGBA_NV21);

Usually, YUV images are 1 channel images with 1.5*height (if it were an RGB or grayscale image).

Or you could create a new Mat and pass jint array to native function and use that array to set pixels of bitmap.

jint *_out = env->GetIntArrayElements(out, 0);     

cv::Mat _yuv(height + height/2, width, CV_8UC1, (uchar*)imagebuffer);
cv::Mat _rgba(height, width, CV_8UC4, (uchar *)_out);

cv::cvtColor(_yuv, _rgba, CV_YUV2RGBA_NV21);

env->ReleaseIntArrayElements(out, _out, 0);

In Java,

bitmap = Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_8888);
pixels = new int[width * height];

native_function(height, width, bytedata, pixels);

bitmap.setPixels(pixels, 0, width, 0, 0, width, height);