MediaCodec KEY_FRAME_RATE seems to be ignored

Question

I am trying to modify the source for screenrecord in android 4.4 and lower the captured frame rate, but no matter what value I put in:

format->setFloat("frame-rate", 5);

the result is always the same ( a very high frame rate )

Is the encoder ignoring this property ? how can I control the frame rate ?

Answer 1

The frame-rate value is not ignored, but it doesn't do what you want.

The combination of frame-rate and i-frame-interval determines how often I-frames (also called "sync frames") appear in the encoded output. The frame rate value might also play a role in meeting the bitrate target on some devices, but I'm not sure about that (see e.g. this post).

The MediaCodec encoder does not drop frames. If you want to reduce the frame rate, you have to do so by sending fewer frames to it.

The screenrecord command doesn't "sample" the screen at a fixed frame rate. Instead, every frame it receives from the surface compositor (SurfaceFlinger) is sent to the encoder, with an appropriate time stamp. If screenrecord receives 60 frames per seconds, you'll have 60fps output. If it receives 10 frames in quick succession, followed by nothing for 5 seconds, followed by a couple more, you'll have exactly that in the output file.

You can modify screenrecord to drop frames, but you have to be a bit careful. If you try to reduce the maximum frame rate from 60fps to 30fps by dropping every-other frame, you run the risk that in a "frame0 - frame1 - long_pause - frame2" sequence you'll drop frame1, and the video will hold on frame0 instead, showing a not-quite-complete animation. So you need to buffer up a frame, and then encode or drop frame N-1 if the difference in presentation times between that and frame N is ~17ms.

The tricky part is that screenrecord, in its default operating mode, directs the frames to the encoder without touching them, so all you see is the encoded output. You can't arbitrarily drop individual frames of encoded data, so you really want to prevent the encoder from seeing them in the first place. If you use the screenrecord v1.1 sources you can tap into "overlay" mode, used for --bugreport, to have the frames pass through screenrecord on their way to the encoder.

In some respects it might be simpler to write a post-processor that reduces the frame rate. I don't know how much quality would be lost by decoding and re-encoding the video.

Update: for an example of how to do it crudely, add this to processFrame_l():

     int64_t droppedFrames = 0;
+    {
+        static int flipflop = 0;
+        flipflop = 1 - flipflop;
+        if (flipflop) {
+            printf("dropping frame %lld\n", frameNumber);
+            return;
+        }
+    }

     if (mLastFrameNumber > 0) {

Note this comes after updateTexImage(), which acquires the next buffer, and skips the call to swapBuffers(), which submits the buffer to the video encoder.