What is the use of presentationTimeUs for MediaCodec?

Question

The official definition for presentationTimeUs in queueInputBuffer (int index, int offset, int size, long presentationTimeUs, int flags) is the following:

The presentation timestamp in microseconds for this buffer. This is normally the media time at which this buffer should be presented (rendered).

Why does the decoder need this if it is up to the app when to present the decoded image? I have tried some arbitrary numbers for presentationTimeUs, and they do not seem to make any difference to the decoding. For example, if I double the original values of presentationTimeUs, the video seems to be decoded exactly the same way and at the same speed as the original.

Could anyone shed some light on this?

Answer 1

The decoder needs to know the input buffer's timestamps for multiple reasons.

First, if the stream has B-frames, then the reordering of buffers and assigning the correct timestamps to the buffers is performed by the decoder. So when the timestamps are received on the input buffer, the same is queued for reordering.

Secondly, if the use-case is something like Android-TV which infact has the tunneled video playback, the timestamp is consumed by the video decoder which is tunneled to the underlying HW block for synchronization and rendering.

Lastly, if there is any drop in packets or frames, the decoder can potentially perform some sort of concealment if it an observes abrupt jump in timestamps without a flush being called. This is not a norm, but is an advanced feature of some decoders.

In traditional cases, as you have pointed out, the synchronization is performed by the player engine in which decoder shall reflect the input buffer timestamp onto the output buffer.