Android Precise seeking of video

Question

I'm struggling with precise seeking using MediaExtractor's seekTo(). While I can seek to sync frames without problems, I would like to seek to specific time. This question led me to some ideas how to do this, but I'm not sure if they are valid. Basicly, I would have to seek to closest previous sync frame and then advance() the extractor until target time is reached. Every frame in the process would be fed to the decoder, i.e the first I-frame and the rest P-frames. This is related code snippet (based on google/grafika's MoviePlayer):

extractor.seekTo((long) seekTarget[threadNr], MediaExtractor.SEEK_TO_PREVIOUS_SYNC);

...

while (extractor.getSampleTime() < (long) seekTarget[threadNr]) {
    Log.d(TAG, "Thread " + threadNr + " advanced to timestamp " + extractor.getSampleTime());

    int inputBufIndex = decoder.dequeueInputBuffer(TIMEOUT_USEC);
    if (inputBufIndex >= 0) {
        ByteBuffer inBufer = decoderInputBuffers[inputBufIndex];
        int chunkSize = extractor.readSampleData(inBufer, 0);

        if (chunkSize < 0) {
            // End of stream -- send empty frame with EOS flag set.
            decoder.queueInputBuffer(inputBufIndex, 0, 0, 0L,
                    MediaCodec.BUFFER_FLAG_END_OF_STREAM);
            inputDone = true;
            if (VERBOSE) Log.d(TAG, "sent input EOS");
        } else {
            if (extractor.getSampleTrackIndex() != trackIndex) {
                Log.w(TAG, "WEIRD: got sample from track " +
                        extractor.getSampleTrackIndex() + ", expected " + trackIndex);
            }

            long presentationTimeUs = extractor.getSampleTime();
            decoder.queueInputBuffer(inputBufIndex, 0, chunkSize,
                    presentationTimeUs, 0 /*flags*/);
            if (VERBOSE) {
                Log.d(TAG, "submitted frame " + inputChunk + " to dec, size=" +
                        chunkSize + " inputBufIndex: " + inputBufIndex);
            }
            inputChunk++;
            extractor.advance();
        }
    }
}

As you can imagine, usually I'm queuing large amount of frames, but for now I'm fine with memory consumption or eventual lag. The problem is that the dequeueInputBuffer() method works only for some time in the loop, eventualy stucks at returning -1, which accordingly to documentation means that the buffer is unavailiable. If I would change the TIMEOUT_USEC to -1, I get infinite loop.

Can someone please tell me if this approach is correct or why at some point I can't get access to inputBuffer?

Answer 1

You don't appear to be pulling buffers from the output side. The MediaCodec decoder doesn't drop frames, so when its internal buffers fill up it will stop handing you input buffers.

You need to drain the decoder by requesting output buffers. When you release the buffer, set the "render" flag to false so it doesn't appear on screen.