Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to reuse the Surface of MediaRecorder in the camera2 API on Android?

So I have played around all weekend with the camera2 api. Now I'm at a point where I begin to understand how things are wired together. While testing the api to implement a video recording app I hit a wall though.

I started by changing the Android Camera2Video Sample to my needs. What bugged me is that after each recording process the camera session is being recreated. Even worse, when a recording session is beeing started whats happening is that the preview session will be destroyed first and a recording session is created. After the recording session is done it gets destroyed and a new preview session is created.

The documentation clearly states:

Creating a session is an expensive operation and can take several hundred milliseconds... CameraCaptureSession Documentation

The result looks pretty ugly and the screen stutters when I hit record and stop. I wanted to improve this behavior so I fiddled around with the code.

What I do now is I create my one and only CameraSession where I add my preview surface (a TextureView) and also the Surface from a already created MediaRecorder by calling its getSurface method. This works fine for the first video but when I try to capture a second Video I get a IllegalArgumentException: Bad argument passed to camera service. I think this is because the surface of the MediaRecorder which I pass to the CameraSession upon it's creation is somehow destroyed or changed when I reset the MediaRecorder to prepare a new recording.

My question now is, is there any way around this problem? (setInputSurface(Surface surface)) might be but the api level is too high so I didn't test it.

Here is a quick overview over the relevant code pieces:

  1. setup the MediaRecorder

    private void setUpMediaRecorder() throws IOException {
        if (mMediaRecorder == null) {
            mMediaRecorder = new MediaRecorder();
        }
        mMediaRecorder.setVideoEncodingBitRate(5000000);
        mMediaRecorder.setAudioSource(MediaRecorder.AudioSource.MIC);
        mMediaRecorder.setVideoSource(MediaRecorder.VideoSource.SURFACE);
        mMediaRecorder.setOutputFormat(MediaRecorder.OutputFormat.MPEG_4);
        mMediaRecorder.setVideoFrameRate(24);
        mMediaRecorder.setVideoSize(mVideoSize.getWidth(), mVideoSize.getHeight());
        mMediaRecorder.setVideoEncoder(MediaRecorder.VideoEncoder.H264);
        mMediaRecorder.setAudioEncoder(MediaRecorder.AudioEncoder.AAC);
        mMediaRecorder.setOrientationHint(SENSOR_ORIENTATION_DEFAULT_DEGREES);
        mNextVideoAbsolutePath = getVideoFilePath();
        mMediaRecorder.setOutputFile(mNextVideoAbsolutePath);
        mMediaRecorder.prepare();
    }
    
  2. create the all mighty recording session

    SurfaceTexture texture = mTextureView.getSurfaceTexture();
    texture.setDefaultBufferSize(mPreviewSize.getWidth(), mPreviewSize.getHeight());
    
    List<Surface> surfaces = new ArrayList<>();
    // Set up Surface for the camera preview
    mPreviewSurface = new Surface(texture);
    surfaces.add(mPreviewSurface);
    
    // Set up Surface for the MediaRecorder
    mRecorderSurface = mMediaRecorder.getSurface();
    surfaces.add(mRecorderSurface);
    
    // create the capture session
    mCameraDevice.createCaptureSession(surfaces, new CameraCaptureSession.StateCallback() {
    
        @Override
        public void onConfigured(@NonNull CameraCaptureSession cameraCaptureSession) {
            mCameraSession = cameraCaptureSession;
    
            // now that the session is created, start using it  for the preview
            showPreview();
        }
    
        @Override
        public void onConfigureFailed(@NonNull CameraCaptureSession cameraCaptureSession) {
            ....
                }
            }
        }, mBackgroundHandler);
    } catch (CameraAccessException) {
        e.printStackTrace();
    }
    
    void showPreview() {
        mPreviewBuilder = mCameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_PREVIEW);
        mPreviewBuilder.set(CaptureRequest.CONTROL_MODE, CameraMetadata.CONTROL_MODE_AUTO);
        mPreviewBuilder.addTarget(mPreviewSurface);
        mCameraSession.setRepeatingRequest(mPreviewBuilder.build(), null, mBackgroundHandler);
    }
    
  3. start recording a video

    mVideoBuilder = mCameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_RECORD);
    mVideoBuilder.set(CaptureRequest.CONTROL_AF_MODE, CaptureRequest.CONTROL_AF_MODE_CONTINUOUS_VIDEO);
    mVideoBuilder.set(CaptureRequest.CONTROL_AE_MODE, CaptureRequest.CONTROL_AE_MODE_ON);
    mVideoBuilder.addTarget(mPreviewSurface);
    mVideoBuilder.addTarget(mRecorderSurface);
    // set the request for the capture
    mCameraSession.setRepeatingRequest(mVideoBuilder.build(), null, mBackgroundHandler);
    // Start recording
    mMediaRecorder.start();
    
  4. stop recording

    mMediaRecorder.stop();
    mMediaRecorder.reset();
    showPreview();
    setUpMediaRecorder(); // this is key to not get an error from the MediaRecorder
    

All of this works perfect and the video recording starts and stops without any hiccups! It's awesome but when I go back to step 3 (after 4) I get the aforementioned IllegalArgumentException: Bad argument passed to camera service. I keep banging my head against the wall but I cannot find a way around this problem.

Any help is greatly appreciated!

Thanks!

like image 921
grAPPfruit Avatar asked Jan 29 '17 22:01

grAPPfruit


1 Answers

Check out MediaRecorder#setInputSurface(android.view.Surface):

Configures the recorder to use a persistent surface when using SURFACE video source.

I stumbled upon it when trying to figure out how to reuse the MediaRecorder capture surface too. This way, you can set your persistent surface to be one of the output surfaces of your capture session, and you won't have to recreate a capture session just to change the MediaRecorder surface generated from a new prepare() call.

The Google Nexus and Pixel camera apps are able to start and stop video recording without any stutter in the preview, so it's definitely possible to do this somehow.

like image 51
zhicong Avatar answered Oct 14 '22 15:10

zhicong