Android Camera2 getPreviewFrame

Question

I am trying to get the camera frame in preview mode. I am running the sample project from github https://github.com/googlesamples/android-Camera2Basic

The issue I am having is getting the frame in preview mode.

Here is the code:

private CameraCaptureSession.CaptureCallback mCaptureCallback = new CameraCaptureSession.CaptureCallback() {

    private void process(CaptureResult result) {
        switch (mState) {
            case STATE_PREVIEW: {

                //HERE, HOW CAN I RETRIEVE THE CURRENT FRAME?

                break;
            }
            case STATE_WAITING_LOCK: {
               ...
                break;
            }
            case STATE_WAITING_PRECAPTURE: {
             ...
                break;
            }
            case STATE_WAITING_NON_PRECAPTURE: {
             ...
                break;
            }
        }
    }

Another thing I tried to get the frame is setting the mImageReader.setOnImageAvailableListener. I was expecting to be able to get the frame onImageAvailable callback, but onImageAvailable is never called. onPreviewFrame is my own method, I need to pass it the current frame.

  mImageReader = ImageReader.newInstance(largest.getWidth(), largest.getHeight(), ImageFormat.JPEG, /*maxImages*/2);
  mImageReader.setOnImageAvailableListener(mOnImageAvailableListener, mBackgroundHandler);

  private final ImageReader.OnImageAvailableListener mOnImageAvailableListener  = new ImageReader.OnImageAvailableListener() {

    @Override
    public void onImageAvailable(ImageReader reader) {
        mTextureView.onPreviewFrame(reader.acquireNextImage().getPlanes([0].getBuffer().array());
    }

};

What I am doing wrong? Thanks.

Answer 1

The OnImageAvailableListener.onImageAvailable callback is never called when a preview frame is available because the CaptureRequest which was sent to the CameraCaptureSession.setRepeatingRequest() method did not list the ImageReader's Surface as an output target.

You identify what output Surfaces (raw byte buffers, essentially) you want the data of each capture to go to when you send the request to the camera. So to get the "preview frames" to trigger the onImageAvailable() callback and then be sent to your onPreviewFrame() method, simply add the line:

mPreviewRequestBuilder.addTarget(mImageReader.getSurface());

This line can go, e.g., after the other similar line that adds the SurfaceTexture's Surface to the same request builder.

Note that this will send every preview frame to your function, as well as the "output frames" from the capture button. You may want some code in the onImageAvailable() callback to discriminate.