Android MediaCodec decode h264 raw frame

Question

I am using Android MediaCodec API to decode h264 frames. I could decode and render the frames on the view. My problem is the decoder miss lots of frames,especially the first some frames. DecodeMediaCodec.dequeueOutputBuffer() return -1. aAbout 150 h264 frames,just decoded 43 frames. I can not find where the problem is. Here is my codes.

 /**
 * init decoder
 */
private void initDecodeMediaCodec()
{
    mDecodeMediaCodec = MediaCodec.createDecoderByType(MIME_TYPE);
    MediaFormat format = MediaFormat.createVideoFormat(MIME_TYPE,
            VIDEO_WIDTH_640,
            VIDEO_HEIGHT_480);

    mDecodeMediaCodec.configure(format,
            new Surface(mRemoteVideoView.getSurfaceTexture()),
            null,
            0);
    mDecodeMediaCodec.start();
    mDecodeInputBuffers = mDecodeMediaCodec.getInputBuffers();
    System.out.println("decode-----"
            + mDecodeMediaCodec.getCodecInfo().getName());
}

After decoder initial,I will start decoder thread.

 /**
 * 
 * @param frameData
 */
private void decode()
{
    new Thread(new Runnable()
    {
        @Override
        public void run()
        {
            while (true)
            {
                ByteBuffer decodeDataBuffer = null;
                try
                {
                    //take h264 frame from cache queue
                    decodeDataBuffer = decodeDataQuene.take();
                }
                catch (InterruptedException e)
                {
                    e.printStackTrace();
                }


                BufferInfo info = new BufferInfo();
                int inputBufferIndex = mDecodeMediaCodec.dequeueInputBuffer(-1);
                System.out.println("inputBufferIndex: " + inputBufferIndex);
                if (inputBufferIndex >= 0)
                {
                    ByteBuffer buffer = mDecodeInputBuffers[inputBufferIndex];
                    buffer.clear();
                    buffer.put(decodeDataBuffer.array());
                    mDecodeMediaCodec.queueInputBuffer(inputBufferIndex,
                            0,
                            decodeDataBuffer.array().length,
                            0,
                            0);
                    decodeDataBuffer.clear();
                    decodeDataBuffer = null;
                }

                int outputBufferIndex = mDecodeMediaCodec.dequeueOutputBuffer(info,
                       1000);
                System.out.println("outputBufferIndex: "
                        + outputBufferIndex);
                do
                {

                    if (outputBufferIndex == MediaCodec.INFO_TRY_AGAIN_LATER)
                    {
                        //no output available yet
                    }
                    else if (outputBufferIndex == MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED)
                    {
                        //encodeOutputBuffers = mDecodeMediaCodec.getOutputBuffers();
                    }
                    else if (outputBufferIndex == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED)
                    {
                        MediaFormat formats = mDecodeMediaCodec.getOutputFormat();
                        //mediaformat changed
                    }
                    else if (outputBufferIndex < 0)
                    {
                        //unexpected result from encoder.dequeueOutputBuffer
                    }
                    else
                    {
                        mDecodeMediaCodec.releaseOutputBuffer(outputBufferIndex,
                                true);

                        outputBufferIndex = mDecodeMediaCodec.dequeueOutputBuffer(info,
                                0);
                        System.out.println("inner outputBufferIndex: "
                                + outputBufferIndex);
                    }
                } while (outputBufferIndex > 0);
            }
        }
    }).start();
}

Anybody knows why?I hope your help.My android device is nexus 7.

Answer 1

Getting -1 back from MediaCodec#dequeueOutputBuffer() is normal. It just means it doesn't have any output ready yet.

It's not the case that you hand MediaCodec a buffer of encoded data and immediately get a decoded buffer back. You hand it a buffer of data, which gets sent to the mediaserver process, which feeds it into the hardware AVC decoder, which may still be initializing or maybe just likes to sit on a few frames. When the decoding process completes, the decoded data gets passed back through mediaserver to your app process.

The trick is, the queueInputBuffer() call returns immediately. In normal operation the input side of the decoder will run several frames ahead of the output side. When you're done feeding input you set the end-of-stream flag, and when you see EOS set on the output you know you've reached the end.

You can find various working examples on bigflake and in Grafika. The DecodeEditEncodeTest and EncodeDecodeTest examples work exclusively with raw H.264, the others use MediaExtractor and MediaMuxer to handle MP4 file wrappers.