mpeg 2 decoding

Question

I want to understand how video and audio decoding works, specially the timing synchronization (how to get 30fps video, how to couple that with audio, etc.). I don't want to know ALL the details, just the essence of it. I want to be able to write a high level simplification of an actual video/audio decoder.

Could you provide pointers to me? An actual C/C++ source code of a MPEG2 video/audio decoder would be the fastest way to understand those things I think.

Answer 1

For audio/video synchronization, basically, every video and audio frame should be time-stamped. The timestamp is typically known as PTS (Presentation Time Stamp). Once a video/audio is decoder by decoder, the audio/video renderer should schedule the frame to be displayed at the right time so that audio/video is synchronized.

I think you can refer to chapter "Timing Model" of MPEG2 Tutorial for details.

Answer 2

Reading source code from a codec that works seems the right way to go. I suggest the following :

http://www.mpeg.org/MPEG/video/mssg-free-mpeg-software.html

Given that it's mentionned on the mpeg.org website, i'd say you'll find what you need here.

In the past i've had some time to work on decoding mpeg videos (no audio though), and the principles are quite simple. There are some pure images included, some intermediary images that are described relatively to the closest main ones, and the rest are described using the closest main/intermediary images.

One time slot, one image. But recent codecs are much more complicated, I guess !

EDIT : synchronization

I am no expert in synchronizing audio and video, but the issue seems to be dealt with using a sync layer (see there for a definition).

Answer 3

You can browse source code of ffmpeg (available through svn), or its API documentation.