How to encode video with transparent background

Question

I am encoding a video using cocoa for OSX (with AVAssetWriter) in h264. This is the configuration:

// Configure video writer
AVAssetWriter *m_videoWriter = [[AVAssetWriter alloc] initWithURL:[NSURL fileURLWithPath:@(outputFile)] fileType:AVFileTypeMPEG4 error:NULL];

// configure video input
NSDictionary *videoSettings = @{ AVVideoCodecKey : AVVideoCodecH264, AVVideoWidthKey : @(m_width), AVVideoHeightKey : @(m_height) };
AVAssetWriterInput* m_writerInput = [[AVAssetWriterInput alloc] initWithMediaType:AVMediaTypeVideo outputSettings:videoSettings];

// Add video input into video writer
[m_videoWriter addInput:m_writerInput];

// Start video writer
[m_videoWriter startWriting];
[m_videoWriter startSessionAtSourceTime:kCMTimeZero];

I am using a 'AVAssetWriterInputPixelBufferAdaptor' element to add frames to the composition like this:

AVAssetWriterInputPixelBufferAdaptor *m_pixelBufferAdaptor = [AVAssetWriterInputPixelBufferAdaptor assetWriterInputPixelBufferAdaptorWithAssetWriterInput:m_writerInput sourcePixelBufferAttributes:NULL];

uint8_t* videobuffer = m_imageRGBA.data;


CVPixelBufferRef pixelBuffer = NULL;
CVReturn status = CVPixelBufferCreate (NULL, m_width, m_height, kCVPixelFormatType_32ARGB, NULL, &pixelBuffer);
if ((pixelBuffer == NULL) || (status != kCVReturnSuccess))
{
    NSLog(@"Error CVPixelBufferPoolCreatePixelBuffer[pixelBuffer=%@][status=%d]", pixelBuffer, status);
    return;
}
else
{
    uint8_t *videobuffertmp = videobuffer;
    CVPixelBufferLockBaseAddress(pixelBuffer, 0);
    GLubyte *pixelBufferData = (GLubyte *)CVPixelBufferGetBaseAddress(pixelBuffer);

    //printf("Video frame pixel: %d, %d, %d, %d\n", videobuffertmp[0], videobuffertmp[1], videobuffertmp[2], videobuffertmp[3]);

    for( int row=0 ; row<m_width ; ++row )
    {
        for( int col=0 ; col<m_height ; ++col )
        {
            memcpy(&pixelBufferData[0], &videobuffertmp[3], sizeof(uint8_t));       // alpha
            memcpy(&pixelBufferData[1], &videobuffertmp[2], sizeof(uint8_t));       // red
            memcpy(&pixelBufferData[2], &videobuffertmp[1], sizeof(uint8_t));       // green
            memcpy(&pixelBufferData[3], &videobuffertmp[0], sizeof(uint8_t));       // blue

            pixelBufferData += 4*sizeof(uint8_t);
            videobuffertmp  += 4*sizeof(uint8_t);
        }
    }

    CVPixelBufferUnlockBaseAddress(pixelBuffer, 0);
}

// transform new frame into pixel buffer
[m_pixelBufferAdaptor appendPixelBuffer:pixelBuffer
                   withPresentationTime:CMTimeMake(m_frameNumber, m_framesPerSecond)];

CFRelease(pixelBuffer);
pixelBuffer = nil;

In the pixel data, the alpha value for the pixels are defined as transparent, but the video has not transparent regions.

I am not sure if the encoder is ignoring the alpha value, or is not possible to encode a video with transparent regions. Is there any way to include the alpha channel values in the encoding process?

Answer 1

You probably can't, at least in H.264. See: How to create an h264 video with an alpha channel for use with HTML5 Canvas?

I guess the transparency can be used in mixing and effects processes before encoding to H.264 but not in the final output.

One workarounds may be setting the transparent areas to a pure green value and using that colour as a matte in a later video editing process (like when they use a green background for the weather forecast). Obviously that only works if the output is intended for such an editing process rather than as final output.