Fastest way to draw a MediaCodec-decoded video frame to screen?

Question

I'm looking for the fastest way to take an image frame received from the MediaCodec decoder and draw it to the Android device screen. The important constraints and explanations are:

1. Cannot use MediaPlayer. No intermediate app allowed.

2. Must draw output frames from the MediaCodec decoder to the screen as quickly as possible (minimize latency).

3. The available decoder output formats are as follows:
   ColorFormat[0] 0x00000013 COLOR_FormatYUV420Planar
   ColorFormat[1] 0x00000015 COLOR_FormatYUV420SemiPlanar
   ColorFormat[2] 0x7F000001 OMX_SEC_COLOR_FormatNV12TPhysicalAddress
   ColorFormat[3] 0x7FC00002 OMX_SEC_COLOR_FormatNV12Tiled

4. The video resolution, and thus the resolution of each output frame, is 960x720.

5. The target platform is Galaxy Note II and the approach can be specific to that platform (e.g. take advantage of available hardware functionality). This does not need to work on other platforms or be a generic solution.

An approach that takes less than 66ms would be good. Less than 33ms would be great. My current approach takes 80-90ms, which sucks. (I won't bother describing it since I don't want to skew the answers in any particular direction.)

Answer 1

Your best bet is to decode directly to a Surface. Decoding to a ByteBuffer is going to slow you down quite a bit. A number of examples on bigflake (e.g. ExtractMpegFramesTest) send the output of a decoder to an off-screen surface and examine it with GLES, but it's a simple change to make it work with an on-screen SurfaceView.

Update: Grafika has two different MediaCodec-based video players that send the output to SurfaceView and TextureView, respectively.