iOS7 AVMutableVideoCompositionLayerInstruction causes video frame to freeze

Question

I'm working with modifying some video via AVMutableVideoCompositionLayerInstruction in the iOS7 SDK.

The following code used to work on iOS 6.1.3, but in iOS7 the video is frozen on the first frame (though I can still hear the audio ok). I got rid of all actual transformations I was applying to verify that adding a video composition alone causes problems.

AVURLAsset* videoAsset = [[AVURLAsset alloc] initWithURL:inputFileURL options:NULL];
AVAssetTrack *videoAssetTrack = [[videoAsset tracksWithMediaType:AVMediaTypeVideo] objectAtIndex:0];

AVMutableVideoCompositionLayerInstruction *layerInstruction =
[AVMutableVideoCompositionLayerInstruction videoCompositionLayerInstructionWithAssetTrack:videoAssetTrack];

AVMutableVideoComposition *mainComposition = [AVMutableVideoComposition videoComposition];
AVMutableVideoCompositionInstruction *mainInstruction = [AVMutableVideoCompositionInstruction videoCompositionInstruction];
mainInstruction.layerInstructions = [NSArray arrayWithObject:layerInstruction];
mainInstruction.timeRange = CMTimeRangeMake(kCMTimeZero, videoAsset.duration);
mainComposition.instructions = [NSArray arrayWithObject:mainInstruction];
mainComposition.frameDuration = videoAsset.duration;
mainComposition.renderSize = CGSizeMake(320, 320);

...
exportSession.videoComposition = mainComposition;

If I do not set the videoComposition attribute of exportSession then the video records ok, but I cannot apply any transformations. Anyone know what could be causing this?

Thanks.

Answer 1

A good way to debug issues with the video composition is to use [AVMutableVideoComposition videoCompositionWithPropertiesOfAsset:asset]. The returned AVMutableVideoComposition should work correctly. Then you can compare the contents of the instructions array with your instructions.

To increase the confusion levels, the asset there can also be an AVComposition. I think the AVFoundation team didn't do the best job when naming these things....

Answer 2

I've been struggling as well with AVMutableVideoCompositionLayerInstruction and mix video with CALayers. After a few days trying different ways what I realise is that the time of the assets are pretty important. The proper way to find out the time of each asset is use the property:

loadValuesAsynchronouslyForKeys:@[@"duration"]

    //Asset url
    NSURL *assetUrl = [NSURL fileURLWithPath:_firstVideoFilePath];

   //audio/video assets
   AVURLAsset * videoAsset = [[AVURLAsset alloc]initWithURL:assetUrl options:nil];

   //var to store the duration
   CMTime __block durationTime;

   //And here we'll be able to proper get the asset duration
   [videoAsset loadValuesAsynchronouslyForKeys:@[@"duration"] completionHandler: ^{
         Float64 durationSeconds = CMTimeGetSeconds([videoAsset duration]);
         durationTime = [videoAsset duration];
         //At this point you have the proper asset duration value, you can start any video processing from here. 
  }];

Hope this helps to anyone with the same issue.