Android SDK: Get raw preview camera image without displaying it

Question

I want to do image processing with a raw image without showing it on the screen, which obviously reduces performance.

According to the answers to this thread Taking picture from camera without preview it was not possible in Android 1.5, but does anybody know if it is possible in Android 4 (API level 15)?

Answer 1

In Android 4, the simplest option to receiving raw image data without displaying it on screen is to use the Camera.setPreviewTexture() call to route the preview frames to the GPU.

You can use this in two ways:

1. Do your actual processing on the GPU: Set up an OpenGL context (OpenGL ES 2 tutorial), and create a SurfaceTexture object in that context. Then pass that object to setPreviewTexture, and start preview. Then, in your OpenGL code, you can call SurfaceTexture.updateTexImage, and the texture ID associated with the SurfaceTexture will be updated to the latest preview frame from the camera. You can also read back the RGB texture data to the CPU for further processing using glReadPixels, if desired.
2. Do your processing on the CPU: You can simply create a dummy SurfaceTexture object without any OpenGL context set up. Pass any integer you want as the texture ID, and connect the SurfaceTexture to the camera using setPreviewTexture. As long as you don't call updateTexImage, the SurfaceTexture will simply discard all data passed into it by the camera. Then set up preview callbacks using setPreviewCallback, and use that data (typically in a YUV format) for CPU processing. This is probably less efficient than #1, but does not require knowing OpenGL.

Answer 2

Since I am not allowed to comment. Regarding Eddy's answer. You need to work with this in the NDK as using the Java interface will negate any performance benefit. Having to work with a PixelBuffer is absolutely insane from a performance standpoint. Your conversion from RGBA888 to YUV also needs to be done in C.

Do not try using a TextureView as is will be even worse. You would have to copy the Pixels into a Bitmap then from a Bitmap into an Array all before the conversion to YUV. This, by itself, takes almost 30% of the cpu utilization on a brand spanking new Nexus 7 2013.

The most efficient way is to talk to Camera.h directly and bypass all of the Android APIs. You can create your own buffer and intercept the YUV data before it goes anywhere else.

Answer 3

Showing preview on the screen does not have performance consequences. On all devices I met, the camera output is "wired" to a surface or texture with no CPU involved, all color conversion and scaling taken care of by dedicated hardware.

There may be other reasons to "hide" the preview, but keep in mind that the purpose of the API initially was to make sure that the end user sees whatever arrives from the camera to the application, for privacy and security reasons.