What is the difference between byte stream and packetized stream in gstreamer rtsp h264 depayloader

Question

In the gstreamer rtp h264 depayloader, there is a check to see if the incoming stream is a byte stream or packetized stream.

Can anybody tell me what is the difference between these two formats?

Also, for the bytestream, the codec_data does not get written to the caps. Any idea why this would be?

Answer 1

H.264 (NAL) Byte Stream

Is used mainly to be sent directly to the decoder on a single PC, and not to be transmitted over a network. It has simple format rules:

• Each frame starts with the same 3 byte start code 0x000001.
• Byte stream must start with Sequence Parameter Sets frame, followed by Picture Parameter Sets frame, then other frames (I, P, B) can follow.
• All frames in it are whole frames – if IDR frame is 10 MB in size, it will be 10 MB in size from its 0x000001 start code, to the next frame's 0x000001 start code.

H.264 Packetized Stream

It is used only to be transmitted over TCP on a limited MTU network. Each network has MTU (Maximum Transmission Unit) that can be sent at a time through TCP. Usually it is around 1500 bytes. So, if you want to send 10 MB IDR frame over TCP, you will have to break it apart so the parts fit the MTU. H.264 Stream that is adopted in this way is called Packetized Stream.

In order to decode this stream, you must reconstruct whole frames on the receiving side, and you usually then want to make H264 NAL Byte Stream out of it, so you can send it to a decoder...

Rules of packetization can be found here: http://www.rfc-editor.org/rfc/rfc3984.txt