Android MediaCodec and camera: how to achieve a higher frame rate to get frame raw data from camera?

Question

The example of CameraToMpegTest.java in bigflake.com, or "Show + capture camera" in Grafika, uses the camera.preview to get the frame data. Experiments show that (Nexus 4, Android 4.4.2) the frame rate is 10 fps. This is not as high as expected.

If we use the same device (Nexus 4, Android 4.4.2) to record a video using the camera, the frame rate is 30fps.

So I assume the lower frame rate using camera.preview lies in the method (preview method). I once read a post, saying that camera preview method has a lower frame rate.

So it seems the workaround is to use raw frame data from the camera hardware directly. How to do that? I have an impression iOS has video processing APIs to do that, directly getting frame raw data from camera. (But I do not know what their frame rate is).

Answer 1

The camera API has two different parameters for controlling the frame rate: setPreviewFrameRate, which takes a single frame rate value and is deprecated, and setPreviewFpsRange, which takes a range of FPS values, and is the currently-recommended control.

The reason the single-FPS setting control is not sufficient is that sometimes you want the camera to slow down its frame rate in dark conditions to keep the viewfinder bright (this is the case for a still camera viewfinder), and sometimes you want the camera to maintain a steady 30fps no matter what (in case of video recording). A single value can't capture which you prefer.

So, the ideal solution is to call getSupportedPreviewFpsRange to get a list of valid FPS ranges the camera supports, and pick the one that's best for your use case. If you're looking for steady 30fps operation, you'd want (30, 30) as the range.

Unfortunately, the set of supported FPS ranges isn't as well tested as it should be, and it's not guaranteed that (30, 30) is on the list. In that case, an alternative is to try the deprecated single-FPS control with a parameter of 30, and turn on the recording hint parameter. This parameter tells the camera device that you're doing recording-like operation, which may switch it to doing a steady frame rate at 30. Unfortunately that's not guaranteed, since it's just a hint.

So in short, to get steady 30fps operation:

1. Query getSupportedPreviewFpsRange
2. If (30,30) is listed, use setPreviewFpsRange(30, 30). This should be enough to guarantee steady frame rate.
3. If not, query getSupportedPreviewFrameRates (30 should always be listed here, but best to double-check)
4. Use setPreviewFrameRate(30) and setRecordingHint(true). This maximizes the likelihood of seeing 30fps operation. But some devices may still not do what you want, unfortunately.

Going forward, we'll hopefully add a requirement that (30, 30) is always listed as a supported range, to simplify this and guarantee steady-state operation.