Low FPS with Camera2 API

Question

I am experiencing a low FPS rate with the new camera2 api. Specifically, despite choosing the highest FPS range (30,30) from

characteristics.get(CameraCharacteristics.CONTROL_AE_AVAILABLE_TARGET_FPS_RANGES);

and setting it with

mPreviewRequestBuilder.set(CaptureRequest.CONTROL_AE_TARGET_FPS_RANGE, bestFPSRange);

I get much lower average rates for high image resolutions on both devices I have tested (Samsung S5 and Xperia Z3 Compact). Here is how I set up the OnImageAvailableListener:

int format = ImageFormat.YUV_420_888;
Size largest = map.getOutputSizes(format)[0];
Log.d("Images", "sizes: " + Arrays.toString(map.getOutputSizes(format)));
mImageReader = ImageReader.newInstance(largest.getWidth(), largest.getHeight(), format,
        /* maxImages */50);
mImageReader.setOnImageAvailableListener(new OnImageAvailableListener() {

    private int  frames      = 0;
    private long initialTime = SystemClock.elapsedRealtimeNanos();

    @Override
    public void onImageAvailable(ImageReader reader) {
        reader.acquireLatestImage().close();
        frames++;
        if ((frames % 30) == 0) {
            long currentTime = SystemClock.elapsedRealtimeNanos();
            long fps = Math.round(frames * 1e9 / (currentTime - initialTime));
            Log.d("Image", "frame# : " + frames + ", approximately " + fps + " fps");
            frames = 0;
            initialTime = SystemClock.elapsedRealtimeNanos();
        }
    }
}, mBackgroundHandler);

Basically, the if statement above is taking an average FPS every 30 samples. In practice, at the highest resolutions on both devices (1920x1080), I see fps ranges of 15-20 fps. Others online seem to indicate, however, that 30fps should be possible regardless of the resolution, and in fact, using the old deprecated camera API, I CAN get 30fps on both devices with the highest resolution. So what am I missing?

I already tried all of the combinations of TEMPLATE (e.g. TEMPLATE_PREVIEW) and format (e.g. ImageFormat.YUV_420_888). Which other knob am I forgetting to twist?

Answer 1

The S5 and Z3 Compact are classified as LEGACY devices. I tested this code on other LEGACY devices, including a

Samsung Galaxy S5

Xperia Z3 Compact

HTC One M9

Huawei Mate S

All of them returned low frame rates (approximately 15fps) at 1080p. One of the phones I found, an LG G4, support the FULL profile. On that device, I was able to get 30fps at even high frame sizes. So I strongly suspect that there is overhead in the camera2 api's wrapper that is causing this problem.