Android video filter

Question

I'm trying to create an app where I am able to add filters to a recorded video. Basically, I want to replicate the functionality that exists in Instagram video, or Viddy.

I've done research and I can't piece it all together. I've looked into using GLSurfaceView to play the recorded video and I know I could use NDK to do the pixel manipulation and send it back to the SurfaceView or save it somehow. The problem is, I don't know how to send the pixel data because there seems to be no function to access it. This idea came from the Camera function "onPreviewFrame". The function returns a byte array allowing me to manipulate the pixels and display it.

Another idea is to use GLSurfaceView and use OpenGL to render the filter. GLSurfaceView has a renderer you can set, but I'm not very familiar with OpenGL. But again, this goes back to actually getting the pixels of each video frame. I also read about ripping each frame as a texture and then manipulating the texture in OpenGL but the answers I've come across are not very detailed.

Lastly, I've looked into JavaCV. Trying to use FFmpegFrameGrabber, but I haven't had much either. I wanted to just grab one frame, but when I try to write the frame's ByteBuffer to an ImageView, I get a "buffer not large enough for pixels" error.

Any guidance would be great.

Answer 1

From Android 4.3 you can use a Surface as the input to your encoder. http://developer.android.com/about/versions/android-4.3.html#Multimedia

So you can use GLSurfaceView and apply the filters using fragment shaders.

You can find some good examples here. http://bigflake.com/mediacodec/