How to understand header of H264

Question

My NAL unit looks like this:

00 00 00 01 67 42 00 1F E6 20 0A 00 B6 60 22 00 00 03 00 02 00 00 03 00 79 C0 00 00 10 FD D0 00 00 40 16 43 F4 A4 18 00 00 03 02 1D 97 00 00 1A B3 F1 FA 52 05 00 00 00 01 68

How can I understand what is the data in RBSP? Like payload size and what parts of the data signifies it.

Answer 1

In the H264 format the byte stream is organised into many NAL unit. In order to understand where a NAL unit starts a three-byte or four-byte start code, 0x000001 or 0x00000001, is placed at the beginning of each NAL unit.

There is the possibility that this sequence is present also in the raw data, in this case an emulation prevention byte 0x03 is used to transform the sequences 0x000000, 0x000001, 0x000002 and 0x000003 into 0x00000300, 0x00000301, 0x00000302 and 0x00000303 respectively.

In each NAL unit the header occupies just the first byte of its sequence, the rest of the byte represents the actual payload.

The header contains information about the type of data contained in the payload, and it can divided in three parts.

The header 0x67 ( which is the header in your NAL unit ) for example corresponds to the binary sequence 0110 0111. The first bit of this sequence ( which is a 0 ) is the forbidden zero and is used to verify if errors where encountered during the transmission of the packet.

The following 2 bits ( the 11 ) are called nal_ref_idc and they indicates if NAL unit is a reference field, frame or picture.

The remaining 5 bits specify the nal_unit_type. It specifies the type of RBSP data structure contained in the NAL unit. For a more detailed explanation of the NAL unit header you can refer to table 7.1 found in here or to the official RFC