NSArray of UIImages to video error has distortion in the output

Question

I am relatively new to programming and although i am ok with normal functions, i am however completely new to video editing

So i have managed to find some code online to do the jobs shown below:

- (void)writeImagesAsMovie:(NSArray *)array {
NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDirectory, YES);
NSString *documentDirectory = [paths objectAtIndex:0];
NSString *saveLocation = [documentDirectory stringByAppendingString:@"/temp.mov"];

if ([[NSFileManager defaultManager] fileExistsAtPath:saveLocation]) {
    [[NSFileManager defaultManager] removeItemAtPath:saveLocation error:NULL];
}

UIImage *first = [array objectAtIndex:0];

CGSize frameSize = first.size;

NSError *error = nil;
AVAssetWriter *videoWriter = [[AVAssetWriter alloc] initWithURL:
                              [NSURL fileURLWithPath:saveLocation] fileType:AVFileTypeQuickTimeMovie
                                                          error:&error];

if(error) {
    NSLog(@"error creating AssetWriter: %@",[error description]);
}
NSDictionary *videoSettings = [NSDictionary dictionaryWithObjectsAndKeys:
                               AVVideoCodecH264, AVVideoCodecKey,
                               [NSNumber numberWithInt:frameSize.width], AVVideoWidthKey,
                               [NSNumber numberWithInt:frameSize.height], AVVideoHeightKey,
                               nil];



AVAssetWriterInput *writerInput = [AVAssetWriterInput
                                    assetWriterInputWithMediaType:AVMediaTypeVideo
                                    outputSettings:videoSettings];

NSMutableDictionary *attributes = [[NSMutableDictionary alloc] init];
[attributes setObject:[NSNumber numberWithUnsignedInt:kCVPixelFormatType_32ARGB] forKey:(NSString*)kCVPixelBufferPixelFormatTypeKey];
[attributes setObject:[NSNumber numberWithUnsignedInt:frameSize.width] forKey:(NSString*)kCVPixelBufferWidthKey];
[attributes setObject:[NSNumber numberWithUnsignedInt:frameSize.height] forKey:(NSString*)kCVPixelBufferHeightKey];

AVAssetWriterInputPixelBufferAdaptor *adaptor = [AVAssetWriterInputPixelBufferAdaptor
                                                 assetWriterInputPixelBufferAdaptorWithAssetWriterInput:writerInput
                                                 sourcePixelBufferAttributes:attributes];

[videoWriter addInput:writerInput];

// fixes all errors
writerInput.expectsMediaDataInRealTime = YES;

//Start a session:
[videoWriter startWriting];

[videoWriter startSessionAtSourceTime:kCMTimeZero];

CVPixelBufferRef buffer = NULL;
buffer = [self pixelBufferFromCGImage:[first CGImage]];
BOOL result = [adaptor appendPixelBuffer:buffer withPresentationTime:kCMTimeZero];

if (result == NO) //failes on 3GS, but works on iphone 4
    NSLog(@"failed to append buffer");

if(buffer) {
    CVBufferRelease(buffer);
}



//int reverseSort = NO;
NSArray *newArray = array;



int fps = 10;


int i = 0;
for (UIImage *image in newArray)
{
    [NSThread sleepForTimeInterval:0.02];
    if (adaptor.assetWriterInput.readyForMoreMediaData) {

        i++;
        CMTime frameTime = CMTimeMake(1, fps);
        CMTime lastTime = CMTimeMake(i, fps);
        CMTime presentTime = CMTimeAdd(lastTime, frameTime);

        UIImage *imgFrame = image;//[UIImage imageWithContentsOfFile:filePath] ;
        buffer = [self pixelBufferFromCGImage:[imgFrame CGImage]];
        BOOL result = [adaptor appendPixelBuffer:buffer withPresentationTime:presentTime];

        if (result == NO) //failes on 3GS, but works on iphone 4
        {
            NSLog(@"failed to append buffer");
            NSLog(@"The error is %@", [videoWriter error]);
            [NSThread sleepForTimeInterval:0.5];
        }

        if(buffer) {
            CVBufferRelease(buffer);
        }


    } else {
        NSLog(@"error");
        i--;
    }
}

//Finish the session:
[writerInput markAsFinished];
[videoWriter finishWriting];
CVPixelBufferPoolRelease(adaptor.pixelBufferPool);

NSLog(@"Movie created successfully");
}

- (CVPixelBufferRef) pixelBufferFromCGImage: (CGImageRef) image
{


    NSDictionary *options = [NSDictionary dictionaryWithObjectsAndKeys:
                             [NSNumber numberWithBool:YES], kCVPixelBufferCGImageCompatibilityKey,
                             [NSNumber numberWithBool:YES], kCVPixelBufferCGBitmapContextCompatibilityKey,
                             nil];
    CVPixelBufferRef pxbuffer = NULL;

    CVPixelBufferCreate(kCFAllocatorDefault, CGImageGetWidth(image),
                        CGImageGetHeight(image), kCVPixelFormatType_32ARGB, (__bridge CFDictionaryRef) options,
                        &pxbuffer);

    CVPixelBufferLockBaseAddress(pxbuffer, 0);
    void *pxdata = CVPixelBufferGetBaseAddress(pxbuffer);

    CGColorSpaceRef rgbColorSpace = CGColorSpaceCreateDeviceRGB();
    CGContextRef context = CGBitmapContextCreate(pxdata, CGImageGetWidth(image),
                                                 CGImageGetHeight(image), 8, 4*CGImageGetWidth(image), rgbColorSpace,
                                                 kCGImageAlphaNoneSkipFirst);

    CGContextConcatCTM(context, CGAffineTransformMakeRotation(0));

    //    CGAffineTransform flipVertical = CGAffineTransformMake(
    //                                                           1, 0, 0, -1, 0, CGImageGetHeight(image)
    //                                                           );
    //    CGContextConcatCTM(context, flipVertical);



    //    CGAffineTransform flipHorizontal = CGAffineTransformMake(
    //                                                             -1.0, 0.0, 0.0, 1.0, CGImageGetWidth(image), 0.0
    //                                                             );
    //
    //    CGContextConcatCTM(context, flipHorizontal);


    CGContextDrawImage(context, CGRectMake(0, 0, CGImageGetWidth(image),
                                           CGImageGetHeight(image)), image);
    CGColorSpaceRelease(rgbColorSpace);
    CGContextRelease(context);

    CVPixelBufferUnlockBaseAddress(pxbuffer, 0);

    return pxbuffer;
}

But the problem that i am having is that the output of the video is some how corrupted (it does play although it has funny lines shown below:

I would be so grateful for any help

Many Thanks

Thomas

Answer 1

I have been seeing problems with the H264 video encoding hardware where it can corrupt input that does not match an known aspect ratio. For example, my testing shows that if one video dimension is smaller than 128 pixel, the video will not encode.

What I have seen working is 128x128, 192x128, 240x160, 480x320, and others.

See this page on aspect ratios

P.S. You will likely want to use the AVAssetWriterInputPixelBufferAdaptor since it contains a pixel buffer pool that you can use via CVPixelBufferPoolCreatePixelBuffer(). Also, you will want to assert(adaptor.pixelBufferPool); after calling startSessionAtSourceTime to ensure that your adaptor can write to the writer.