How to find SPS and PPS string in H264 codec from mp4

Question

As you all know that in Smooth Stream client manifest file, contains a "CodecPrivateData" attribute in video tag. Now after my initial investigation I found that this string is formed by using SPS and PPS which are essentially NAL units.

I am looking for a way to extract that information from video GOP, so that I can use the same to create manifest file and manually substitute codec private data

Basically, i am looking forward to create custom app to create smooth representation using ffmpeg

Answer 1

Note that SPS/PPS are stored separately from video track in the mp4 file in one of the global headers(avcC portion of the global header).

Here is format: 8+ bytes per ISO/IEC 14496-10

                 = long unsigned offset + long ASCII text string 'avcC'
                -> 1 byte version = 8-bit hex version  (current = 1)
                -> 1 byte H.264 profile = 8-bit unsigned stream profile
                -> 1 byte H.264 compatible profiles = 8-bit hex flags
                -> 1 byte H.264 level = 8-bit unsigned stream level
                -> 1 1/2 nibble reserved = 6-bit unsigned value set to 63
                -> 1/2 nibble NAL length = 2-bit length byte size type
                  - 1 byte = 0 ; 2 bytes = 1 ; 4 bytes = 3
                -> 1 byte number of SPS = 8-bit unsigned total
                -> 2+ bytes SPS length = short unsigned length
                -> + SPS NAL unit = hexdump
                -> 1 byte number of PPS = 8-bit unsigned total
                -> 2+ bytes PPS length = short unsigned length
                -> + PPS NAL unit = hexdump

If you just want to extract SPS/PPS from a single .mp4, you can use hex editor and get the SPS/PPS by inspection based on the MP4 format specs above(look for "avcC" string by searching from the end of the file); and then add the SPS/PPS bytes to an c-style array for your use.

Otherwise you can use ffmpeg together with h264bitstream utility to extract SPS/PPS. First run ffmpeg on the command line to extract h264 stream:

ffmpeg -i my_funny_video.mp4 -vcodec copy -vbsf h264_mp4toannexb -an my_funny_video.h264

Then run h264_analyze from the h264bitstream utility:

h264_analyze my_funny_video.h264

which will product a detailed analysis on your SPS/PPS and other NALs.