Extract/modify video frames on Android

Question

I have a video file. I want to get each frame of the video and do some modifications to the frame, such as drawing another bitmap in that, putting some text etc.

Is there any API/framework available to get frames from video in Android? I've done something similar in iOS using their AVFramework.

If it is possible with ffmpeg, I will use some of the opensource NDKs available.

Answer 1

Option A:

You can create a SurfaceTexture object and attach this to the MediaPlayer as follows

myPlayer = new MediaPlayer
...
myRedirectionSurface = new Surface(mySurfaceTexture);
myPlayer->setSurface(myRedirectionSurface);

This way, the player's decoded stream is "redirected" to SurfaceTexture and not the SurfaceView. The OnFrameAvailableListener is called whenever you have a decoded frame available. To access/modify the image, you can use the Surface's lock/unlock methods on myRedirectionSurface.

IMPORTANT NOTE: You need to have API level 14 support to get this working !

Option B:

As you have indicated likelihood of using ffmpeg, you can achieve what you intend to, as you have full access to the decoder's output frames. You can start with RockPlayer's or MoboPlayer's ffmpeg port. But in this option, rendering the video output from NDK is not staright-forward. !