CVPixelBufferPool Error ( kCVReturnInvalidArgument/-6661)

Question

I've implemented previous suggestions with Swift (How to use CVPixelBufferPool in conjunction with AVAssetWriterInputPixelBufferAdaptor in iPhone?), but got stuck with an "kCVReturnInvalidArgument" (error value: -6661) when using CVPixelBufferPoolCreatePixelBuffer as guided.

I'm basically trying to create a movie from images, but as the buffer pool isn't created successfully, I can't append pixel buffers--here is my code for doing this.

Any suggestions are highly appreciated!

import Foundation
import Photos
import OpenGLES
import AVFoundation
import CoreMedia

class MovieGenerator {

    var _videoWriter:AVAssetWriter
    var _videoWriterInput: AVAssetWriterInput
    var _adapter: AVAssetWriterInputPixelBufferAdaptor
    var _buffer = UnsafeMutablePointer<Unmanaged<CVPixelBuffer>?>.alloc(1)


    init(frameSize size: CGSize, outputURL url: NSURL) {

    // delete file if exists
    let sharedManager = NSFileManager.defaultManager() as NSFileManager
    if(sharedManager.fileExistsAtPath(url.path!)) {
        sharedManager.removeItemAtPath(url.path, error: nil)
    }

    // video writer
    _videoWriter = AVAssetWriter(URL: url, fileType: AVFileTypeQuickTimeMovie, error: nil)

    // writer input
    var videoSettings = [AVVideoCodecKey:AVVideoCodecH264, AVVideoWidthKey:size.width, AVVideoHeightKey:size.height]
    _videoWriterInput = AVAssetWriterInput(mediaType: AVMediaTypeVideo, outputSettings: videoSettings)
    _videoWriterInput.expectsMediaDataInRealTime = true
    _videoWriter.addInput(_videoWriterInput)

    // pixel buffer adapter
    var adapterAttributes = [kCVPixelBufferPixelFormatTypeKey:kCVPixelFormatType_32BGRA, kCVPixelBufferWidthKey: size.width,
        kCVPixelBufferHeightKey: size.height,
        kCVPixelFormatOpenGLESCompatibility: kCFBooleanTrue]

    _adapter = AVAssetWriterInputPixelBufferAdaptor(assetWriterInput: _videoWriterInput, sourcePixelBufferAttributes: adapterAttributes)
    var poolCreateResult:CVReturn = CVPixelBufferPoolCreatePixelBuffer(nil, _adapter.pixelBufferPool, _buffer)
    println("pool creation:\(poolCreateResult)")

    _videoWriter.startWriting()
    _videoWriter.startSessionAtSourceTime(kCMTimeZero)

}

func addImage(image:UIImage, frameNum:Int, fps:Int)->Bool {


    self.createPixelBufferFromCGImage(image.CGImage, pixelBufferPtr: _buffer)

    var presentTime:CMTime = CMTimeMake(Int64(frameNum), Int32(fps))
    var result:Bool = _adapter.appendPixelBuffer(_buffer.memory?.takeUnretainedValue(), withPresentationTime: presentTime)

    return result
}

func finalizeMovie(timeStamp: CMTime) {
    _videoWriterInput.markAsFinished()
    _videoWriter.endSessionAtSourceTime(timeStamp)
    _videoWriter.finishWritingWithCompletionHandler({println("video writer finished with status: \(self._videoWriter.status)")})
}

func createPixelBufferFromCGImage(image: CGImage, pixelBufferPtr: UnsafeMutablePointer<Unmanaged<CVPixelBuffer>?>) {

    let width:UInt = CGImageGetWidth(image)
    let height:UInt = CGImageGetHeight(image)

    let imageData:CFData = CGDataProviderCopyData(CGImageGetDataProvider(image))
    let options:CFDictionary = [kCVPixelBufferCGImageCompatibilityKey:NSNumber.numberWithBool(true), kCVPixelBufferCGBitmapContextCompatibilityKey:NSNumber.numberWithBool(true)]

    var status:CVReturn = CVPixelBufferCreate(kCFAllocatorDefault, width, height, OSType(kCVPixelFormatType_32BGRA), options, pixelBufferPtr)
    assert(status != 0,"CVPixelBufferCreate: \(status)")

    var lockStatus:CVReturn = CVPixelBufferLockBaseAddress(pixelBufferPtr.memory?.takeUnretainedValue(), 0)
    println("CVPixelBufferLockBaseAddress: \(lockStatus)")

    var pxData:UnsafeMutablePointer<(Void)> = CVPixelBufferGetBaseAddress(pixelBufferPtr.memory?.takeUnretainedValue())
    let bitmapinfo = CGBitmapInfo.fromRaw(CGImageAlphaInfo.NoneSkipFirst.toRaw())
    let rgbColorSpace:CGColorSpace = CGColorSpaceCreateDeviceRGB()

    var context:CGContextRef = CGBitmapContextCreate(pxData, width, height, 8, 4*CGImageGetWidth(image), rgbColorSpace, bitmapinfo!)

    CGContextDrawImage(context, CGRectMake(0, 0, CGFloat(width), CGFloat(height)), image)

    CVPixelBufferUnlockBaseAddress(pixelBufferPtr.memory?.takeUnretainedValue(), 0)


}



}

Answer 1

I can't exactly answer your question, frustratingly, but I am working on code that does essentially the same thing. And, mine happens to get further than the error you have been getting; it gets all the way to the point where it's attempting to add the images to the movie and then simply fails by never getting a successful result from appendPixelBuffer() -- and I'm not sure how to figure out why. I'm posting this in the hopes that it helps you get further, though.

(My code is adapted from AVFoundation + AssetWriter: Generate Movie With Images and Audio, and I used your post to help navigate som e of the pointer interop shenanigans...)

func writeAnimationToMovie(path: String, size: CGSize, animation: Animation) -> Bool {
    var error: NSError?
    let writer = AVAssetWriter(URL: NSURL(fileURLWithPath: path), fileType: AVFileTypeQuickTimeMovie, error: &error)

    let videoSettings = [AVVideoCodecKey: AVVideoCodecH264, AVVideoWidthKey: size.width, AVVideoHeightKey: size.height]

    let input = AVAssetWriterInput(mediaType: AVMediaTypeVideo, outputSettings: videoSettings)
    let pixelBufferAdaptor = AVAssetWriterInputPixelBufferAdaptor(assetWriterInput: input, sourcePixelBufferAttributes: nil)
    input.expectsMediaDataInRealTime = true
    writer.addInput(input)

    writer.startWriting()
    writer.startSessionAtSourceTime(kCMTimeZero)

    var buffer: CVPixelBufferRef

    var frameCount = 0
    for frame in animation.frames {
        let rect = CGRectMake(0, 0, size.width, size.height)
        let rectPtr = UnsafeMutablePointer<CGRect>.alloc(1)
        rectPtr.memory = rect
        buffer = pixelBufferFromCGImage(frame.image.CGImageForProposedRect(rectPtr, context: nil, hints: nil).takeUnretainedValue(), size)
        var appendOk = false
        var j = 0
        while (!appendOk && j < 30) {
            if pixelBufferAdaptor.assetWriterInput.readyForMoreMediaData {
                let frameTime = CMTimeMake(Int64(frameCount), 10)
                appendOk = pixelBufferAdaptor.appendPixelBuffer(buffer, withPresentationTime: frameTime)
                // appendOk will always be false
                NSThread.sleepForTimeInterval(0.05)
            } else {
                NSThread.sleepForTimeInterval(0.1)
            }
            j++
        }
        if (!appendOk) {
            println("Doh, frame \(frame) at offset \(frameCount) failed to append")
        }
    }

    input.markAsFinished()
    writer.finishWritingWithCompletionHandler({
        if writer.status == AVAssetWriterStatus.Failed {
            println("oh noes, an error: \(writer.error.description)")
        } else {
            println("hrmmm, there should be a movie?")
        }
    })

    return true;
}

Where pixelBufferFromCGImage is defined like so:

func pixelBufferFromCGImage(image: CGImageRef, size: CGSize) -> CVPixelBufferRef {
    let options = [
        kCVPixelBufferCGImageCompatibilityKey: true,
        kCVPixelBufferCGBitmapContextCompatibilityKey: true]
    var pixBufferPointer = UnsafeMutablePointer<Unmanaged<CVPixelBuffer>?>.alloc(1)

    let status = CVPixelBufferCreate(
        nil,
        UInt(size.width), UInt(size.height),
        OSType(kCVPixelFormatType_32ARGB),
        options,
        pixBufferPointer)

    CVPixelBufferLockBaseAddress(pixBufferPointer.memory?.takeUnretainedValue(), 0)

    let rgbColorSpace = CGColorSpaceCreateDeviceRGB()
    let bitmapinfo = CGBitmapInfo.fromRaw(CGImageAlphaInfo.NoneSkipFirst.toRaw())

    var pixBufferData:UnsafeMutablePointer<(Void)> = CVPixelBufferGetBaseAddress(pixBufferPointer.memory?.takeUnretainedValue())

    let context = CGBitmapContextCreate(
        pixBufferData,
        UInt(size.width), UInt(size.height),
        8, UInt(4 * size.width),
        rgbColorSpace, bitmapinfo!)

    CGContextConcatCTM(context, CGAffineTransformMakeRotation(0))
    CGContextDrawImage(
        context,
        CGRectMake(0, 0, CGFloat(CGImageGetWidth(image)), CGFloat(CGImageGetHeight(image))),
        image)

    CVPixelBufferUnlockBaseAddress(pixBufferPointer.memory?.takeUnretainedValue(), 0)
    return pixBufferPointer.memory!.takeUnretainedValue()
}

Answer 2

Per the docs for pixelBufferPool:

This property is NULL before the first call to startSessionAtTime:on the associated AVAssetWriter object.

Moving the call to CVPixelBufferPoolCreatePixelBuffer to the end of init should fix the immediate problem.

A few other observations:

• You have your AVAssetWriterInputPixelBufferAdaptor configured for BGRA, but in createPixelBufferFromCGImage you're using RGB. Your final videos will look strange if the pixel formats are mismatched.
• You don't need to call CVPixelBufferCreate in your createPixelBufferFromCGImage method. This defeats the purpose of using the buffer pool.
• If you're running this in a tight loop, memory consumption will become a problem. Using autoreleasepool and being careful with takeUnretainedValue vs takeRetainedValue will help.

I've posted reference implementations for Swift 1.2, 2.0, and 3.0 that use buffer pools.