Android Software Decoder (OMX.google.h264.decoder) Fails to decode H264 video

Question

I am experiencing issues when trying to extract and decode H264 video using Androids software decoder (OMX.google.h264.decoder). The issue seems to span multiple devices.

This video does playback fine using a Nexus 5 hardware decoder (OMX.qcom.video.decoder.avc).

The sample code below exhibits the issue and is a fairly standard example of using the android MediaCodec and MediaExtractor classes.

The exception I get is an illegal state exception when the first buffer is passed to the decoder.

The video is 720x480 20fps encoded at the Baseline profile so should meet compatibility guidelines.

Here is a video sample to accompany the code

I would greatly appreciate any guidance on getting the software video decoder correctly working with H264 video.

public void doMp4Test()
{
    try
    {
        //String filename = "webserver_h264.mp4";
        String filename = "toodee-720p.mp4";
        MediaExtractor extractor = new MediaExtractor();
        extractor.setDataSource(Constants.RootDirectory + File.separator + filename); 

        MediaCodec decoder = null;

        for (int i = 0; i < extractor.getTrackCount(); i++)
        {
            MediaFormat format = extractor.getTrackFormat(i);
            String mime = format.getString(MediaFormat.KEY_MIME);
            if (mime.startsWith("video/"))
            {
                extractor.selectTrack(i);
                decoder = MediaCodec.createByCodecName("OMX.google.h264.decoder");
                // decoder = MediaCodec.createDecoderByType("OMX.qcom.video.decoder.avc"); // working decoder type
                decoder.configure(format, m_surface, null, 0);
                break;
            }
        }

        if (decoder == null)
        {
            Log.e("DecodeActivity", "Can't find video info!");
            return;
        }

        decoder.start();

        ByteBuffer[] inputBuffers = decoder.getInputBuffers();
        ByteBuffer[] outputBuffers = decoder.getOutputBuffers();
        BufferInfo info = new BufferInfo();
        boolean isEOS = false;
        long startMs = System.currentTimeMillis();

        while (!Thread.interrupted())
        {
            if (!isEOS)
            {
                int inIndex = decoder.dequeueInputBuffer(10000);
                if (inIndex >= 0)
                {
                    ByteBuffer buffer = inputBuffers[inIndex];
                    int sampleSize = extractor.readSampleData(buffer, 0);
                    if (sampleSize < 0)
                    {
                        Log.d("DecodeActivity", "InputBuffer BUFFER_FLAG_END_OF_STREAM");
                        decoder.queueInputBuffer(inIndex, 0, 0, 0, MediaCodec.BUFFER_FLAG_END_OF_STREAM);
                        isEOS = true;
                    }
                    else
                    {
                        int flags = 0;// extractor.getSampleFlags();
                        decoder.queueInputBuffer(inIndex, 0, sampleSize, extractor.getSampleTime(), flags);
                        extractor.advance();
                    }
                }
            }

            int outIndex = decoder.dequeueOutputBuffer(info, 10000);
            switch (outIndex)
            {
            case MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED:
                Log.d("DecodeActivity", "INFO_OUTPUT_BUFFERS_CHANGED");
                outputBuffers = decoder.getOutputBuffers();
                break;
            case MediaCodec.INFO_OUTPUT_FORMAT_CHANGED:
                Log.d("DecodeActivity", "New format " + decoder.getOutputFormat());
                break;
            case MediaCodec.INFO_TRY_AGAIN_LATER:
                Log.d("DecodeActivity", "dequeueOutputBuffer timed out!");
                break;
            default:
                ByteBuffer buffer = outputBuffers[outIndex];
                Log.v("DecodeActivity", "We can't use this buffer but render it due to the API limit, " + buffer);

                // We use a very simple clock to keep the video FPS, or the
                // video
                // playback will be too fast
                while (info.presentationTimeUs / 1000 > System.currentTimeMillis() - startMs)
                {
                    try
                    {
                        Thread.sleep(10);
                    }
                    catch (InterruptedException e)
                    {
                        e.printStackTrace();
                        break;
                    }
                }
                decoder.releaseOutputBuffer(outIndex, true);
                break;
            }

            if ((info.flags & MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0)
            {
                Log.d("DecodeActivity", "OutputBuffer BUFFER_FLAG_END_OF_STREAM");
                break;
            }
        }

          decoder.stop();
          decoder.release();
          extractor.release();
    }
    catch (Exception e)
    {
        e.printStackTrace();
    }   
}

Answer 1

Looking at the video with ffprobe it looks like it is actually high profile rather than base profile:

Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 720x480 [SAR 1:1 DAR 3:2], 515 kb/s, 24.66 fps, 1000k tbr, 1000k tbn, 2000k tbc (default)

As an aside, the video seems to play but just display a black blank screen when playing back on a couple of browsers when I just tried it on a laptop.