AVAssetWriterInput H.264 Passthrough to QuickTime (.mov) - Passing in SPS/PPS to create avcC atom?

Question

I have a stream of H.264/AVC NALs consisting of types 1 (P frame), 5 (I frame), 7 (SPS), and 8 (PPS). I want to write them into an .mov file without re-encoding. I'm attempting to use AVAssetWriter to do this. The documentation for AVAssetWriterInput states:

Passing nil for outputSettings instructs the input to pass through appended samples, doing no processing before they are written to the output file. This is useful if, for example, you are appending buffers that are already in a desirable compressed format. However, passthrough is currently supported only when writing to QuickTime Movie files (i.e. the AVAssetWriter was initialized with AVFileTypeQuickTimeMovie). For other file types, you must specify non-nil output settings.

I'm trying to create CMSampleBuffers out of these NALs and append them to the asset writer input, but I am unable to input the data in a way that yields a well-formed .mov file, and I can't find any clue anywhere on how to do this.

The best result I've gotten so far was passing in the NALs in Annex B byte stream format (in the order 7 8 5 1 1 1....repeating) and playing the result in VLC. Because of this, I know the NALs contain valid data, but because the .mov file did not have an avcC atom and the mdat atom was filled with an Annex B byte stream, QuickTime will not play the video.

Now I'm trying to pass in the NALs with a 4-byte (as specified by the lengthSizeMinusOne field) length field instead of the Annex B delimiter, which is how they're supposed to be packed into the mdat atom as far as I know.

I am at a loss for how to get the asset writer to write an avcC atom. Every sample I append just gets shoved into the mdat atom.

Does anyone know how I can pass raw H.264 data into an AVAssetWriterInput configured for pass through (nil outputSettings) and have it generate a properly formed QuickTime file?

Answer 1

I have submitted a TSI with apple and found the answer. I hope this saves someone time in the future.

The CMSampleBuffers have associated with them a CMFormatDescription, which contains a description of the data in the sample buffer.

The function prototype for creating the format description is as follows:

OSStatus CMVideoFormatDescriptionCreate (
  CFAllocatorRef allocator,
  CMVideoCodecType codecType,
  int32_t width,
  int32_t height,
  CFDictionaryRef extensions,
  CMVideoFormatDescriptionRef *outDesc
);

I learned, from the Apple technician, that I can use the extensions argument to pass in a dictionary containing the avcC atom data.

The extensions dictionary should be of the following form:

[kCMFormatDescriptionExtension_SampleDescriptionExtensionAtoms ---> ["avcC" ---> <avcC Data>]]

The []'s represent dictionaries. This dictionary can potentially be used to pass in data for arbitrary atoms aside from avcC.

Here is the code I used to create the extensions dictionary that I pass into CMVideoFormatDescriptionCreate:

    const char *avcC = "avcC";
    const CFStringRef avcCKey = CFStringCreateWithCString(kCFAllocatorDefault, avcC, kCFStringEncodingUTF8);
    const CFDataRef avcCValue = CFDataCreate(kCFAllocatorDefault, [_avccData bytes], [_avccData length]);
    const void *atomDictKeys[] = { avcCKey };
    const void *atomDictValues[] = { avcCValue };
    CFDictionaryRef atomsDict = CFDictionaryCreate(kCFAllocatorDefault, atomDictKeys, atomDictValues, 1, nil, nil);

    const void *extensionDictKeys[] = { kCMFormatDescriptionExtension_SampleDescriptionExtensionAtoms };
    const void *extensionDictValues[] = { atomsDict };
    CFDictionaryRef extensionDict = CFDictionaryCreate(kCFAllocatorDefault, extensionDictKeys, extensionDictValues, 1, nil, nil);