Recording video not working with Camera2 API on Samsung Note5 after using ImageReader

Question

As the title says, I'm having a problem recording video on the Samsung Note5 using the Camera2 API.

I've adapted my code from the Camera2Video sample, but the difference being that I setup the MediaRecorder using configuration options from the CamcorderProfile class and also that while previewing before starting the video recording I am capturing to an ImageReader as well as rendering the preview to a SurfaceTexture (the sample does not use an ImageReader).

Here's my startVideoCapture function (almost identical to the sample)

private boolean startVideoCapture() {
    if (null == mCameraDevice || !mTextureView.isAvailable() || null == mPreviewSize) {
        debugToast("Can't start video preview");
        return false;
    }
    try {           
        closePreviewSession();
        if(!setUpMediaRecorder())
        {
            debugToast("setUpMediaRecorder failed");
            return false;
        }
        SurfaceTexture texture = mTextureView.getSurfaceTexture();
        assert texture != null;
        texture.setDefaultBufferSize(mPreviewSize.getWidth(), mPreviewSize.getHeight());
        final CaptureRequest.Builder previewRequestBuilder = mCameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_RECORD);
        List<Surface> surfaces = new ArrayList<Surface>();

        Surface previewSurface = new Surface(texture);
        surfaces.add(previewSurface);
        previewRequestBuilder.addTarget(previewSurface);

        Surface recorderSurface = mMediaRecorder.getSurface();
        surfaces.add(recorderSurface);
        previewRequestBuilder.addTarget(recorderSurface);

        mCameraDevice.createCaptureSession(surfaces, new CameraCaptureSession.StateCallback() {

            @Override
            public void onConfigured(CameraCaptureSession cameraCaptureSession) {
                debugToast("onConfigured callback received");
                mCaptureSession = cameraCaptureSession;
                updateVideoPreview(previewRequestBuilder);
            }

            @Override
            public void onConfigureFailed(CameraCaptureSession cameraCaptureSession) {
                debugToast("onConfigureFailed");                    
            }
        }, mCallbacksInterface.getBackgroundHandler());
    } catch (CameraAccessException e) {
        e.printStackTrace();
        debugToast("CameraAccessException");
        return false;
    } catch (IOException e) {
        e.printStackTrace();
        debugToast("IOException");
        return false;
    }
    debugToast("startVideoCapture success");        
    return true;
}

And here's my code where I set up the image reader when starting the preview:

        // We set up a CaptureRequest.Builder with the output Surface.
        CaptureRequest.Builder previewRequestBuilder = mCameraDevice
                .createCaptureRequest(templateType);

        previewRequestBuilder.addTarget(mImageReader.getSurface());
        previewRequestBuilder.addTarget(surface);

        // Here, we create a CameraCaptureSession for camera preview.
        // surface),
        mCameraDevice.createCaptureSession(
                Arrays.asList(mImageReader.getSurface(), surface),
                new CameraCaptureSession.StateCallback() {

        // etc...

It's all pretty standard stuff, and it works fine on the Nexus 5: I can start one capture session for the preview with an ImageReader in the list of capture surfaces, then stop it and start a new one with a MediaRecorder in the surface list and record a video. But, this doesn't work on the Note5. I get a crash when calling createCaptureSession with the MediaRecorder in startVideoCapture:

10-14 14:49:25.991: E/CameraCaptureSession(13566): Session 1: Failed to create capture session; configuration failed
10-14 14:49:26.011: W/System.err(13566): android.hardware.camera2.CameraAccessException: Operation timed out in camera service
10-14 14:49:26.011: W/System.err(13566):    at android.hardware.camera2.utils.CameraBinderDecorator.throwOnError(CameraBinderDecorator.java:118)
10-14 14:49:26.021: W/System.err(13566):    at android.hardware.camera2.utils.CameraBinderDecorator$CameraBinderDecoratorListener.onAfterInvocation(CameraBinderDecorator.java:73)
10-14 14:49:26.021: W/System.err(13566):    at android.hardware.camera2.utils.Decorator.invoke(Decorator.java:81)
10-14 14:49:26.021: W/System.err(13566):    at java.lang.reflect.Proxy.invoke(Proxy.java:393)
10-14 14:49:26.021: W/System.err(13566):    at $Proxy1.waitUntilIdle(Unknown Source)
10-14 14:49:26.021: W/System.err(13566):    at android.hardware.camera2.impl.CameraDeviceImpl.waitUntilIdle(CameraDeviceImpl.java:950)
10-14 14:49:26.021: W/System.err(13566):    at android.hardware.camera2.impl.CameraDeviceImpl.configureStreamsChecked(CameraDeviceImpl.java:399)
10-14 14:49:26.021: W/System.err(13566):    at android.hardware.camera2.impl.CameraDeviceImpl.createCaptureSessionInternal(CameraDeviceImpl.java:561)
10-14 14:49:26.021: W/System.err(13566):    at android.hardware.camera2.impl.CameraDeviceImpl.createCaptureSession(CameraDeviceImpl.java:476)
10-14 14:49:26.021: W/System.err(13566):    at com.example.Camera2Object.startVideoCapture(Camera2Object.java:2262)

If I remove the ImageReader from the preview capture then it works fine.

Whether I close the ImageReader in closePreviewSession or not makes no difference (I'm already calling abortCaptures and close on the preview CaptureSession).

Does anyone know how to solve this?

Edit: something that's possibly related is that when using an ImageReader on this device, closing the CameraDevice in onPause takes a ridiculously long time (up to 6 seconds). I've managed to mostly hide this from the user by doing it in a separate thread, however if onResume is called within 6 seconds of onPause there will be a delay because I have to wait for it to finish closing before I can open it again. Obviously I can't afford a latency of 6 seconds when beginning video recording.

The problem persists even if I do nothing but call acquireLatestImage(), close it and return immediately in the onImageAvailable callback of the ImageReader. It also happens when the capture resolution for the ImageReader is really small. So, it doesn't seem like it's caused by it getting overloaded processing the ImageReader data.

System log covering the period from app start to video recording (and app shutdown) in this Pastebin.

Answer 1

From your pastebin log:

10-15 19:45:32.501: E/Camera3-Device(3151): Camera 0: waitUntilDrainedLocked: Error waiting for HAL to drain: Connection timed out (-110)

This generally means that something went wrong in the guts of the camera HAL - the camera service is waiting for in-flight captures to complete before creating a new session, but some never return. So things time out and an error is returned to the app.

This is a bug in the device-specific camera code, unfortunately, so Samsung would need to fix it.

As a workaround, you could try to stop the repeating request, wait until all the in-flight requests complete (the device switches to the 'ready' state), and then create the new session. Then there are no captures still in progress for the HAL to mishandle.

This will add extra delay and the preview will freeze for a bit longer, but is probably more robust in this case.