Changing AVPlayerItem's videoComposition property (AVMutableVideoComposition) during playback

Question

I am trying to change a video composition's layout (i.e. its component frames' transforms) during playback. It seems that sometimes this works, and the video composition seamlessly changes to the new set of transforms, but other times it just freezes up and stays with the current transforms. There is no status code change on the AVPlayer instance, and there are no errors on either the player or the player item.

Has anybody experienced this before? Any suggestions on why this is happening, or how to get around it, would be appreciated.

Some code is shown below. The important bit is 'playerItem.videoComposition = videoComposition', which here is triggered on tapping the video (for testing purposes).

Another solution to this problem would be to display the videos on separate layers, but it is imperative that the videos are in sync so a composition seems the only way to achieve this.

@implementation VideoView
{
    CGSize _videoSize;
    CMTimeRange _videoFullRange;

    AVMutableCompositionTrack * _compositionTrackVideoA;
    AVMutableCompositionTrack * _compositionTrackVideoB;
}

+ (Class)layerClass
{
    return [AVPlayerLayer class];
}

- (id)initWithFrame:(CGRect)frame
{
    self = [super initWithFrame:frame];
    if ( self )
    {
        NSString * videoAPath = [[NSBundle mainBundle] pathForResource:@"cam09v2" ofType:@"mp4"];
        NSString * videoBPath = [[NSBundle mainBundle] pathForResource:@"cam10v2_b" ofType:@"mp4"];
        AVURLAsset * videoAAsset = [AVURLAsset URLAssetWithURL:[NSURL fileURLWithPath:videoAPath] options:nil];
        AVURLAsset * videoBAsset = [AVURLAsset URLAssetWithURL:[NSURL fileURLWithPath:videoBPath] options:nil];

        AVAssetTrack * videoATrack = [[videoAAsset tracksWithMediaType:AVMediaTypeVideo] lastObject];
        AVAssetTrack * videoBTrack = [[videoBAsset tracksWithMediaType:AVMediaTypeVideo] lastObject];
        AVAssetTrack * audioTrack = [[videoAAsset tracksWithMediaType:AVMediaTypeAudio] lastObject];

        _videoSize = [videoATrack naturalSize];
        CMTime videoDuration = videoAAsset.duration;
        _videoFullRange = CMTimeRangeMake(kCMTimeZero, videoDuration);

        AVMutableComposition *composition = [AVMutableComposition composition];
        AVMutableCompositionTrack * compositionTrackVideoA = [composition addMutableTrackWithMediaType:AVMediaTypeVideo preferredTrackID:kCMPersistentTrackID_Invalid];
        AVMutableCompositionTrack * compositionTrackVideoB = [composition addMutableTrackWithMediaType:AVMediaTypeVideo preferredTrackID:kCMPersistentTrackID_Invalid];
        AVMutableCompositionTrack * compositionTrackAudio = [composition addMutableTrackWithMediaType:AVMediaTypeAudio preferredTrackID:kCMPersistentTrackID_Invalid];

        compositionTrackVideoA.preferredTransform = videoATrack.preferredTransform;

        NSError * error = nil;
        if ( ! [compositionTrackVideoA insertTimeRange:_videoFullRange ofTrack:videoATrack atTime:kCMTimeZero error:&error] )
            NSLog(@"%@", error);

        if ( ! [compositionTrackVideoB insertTimeRange:_videoFullRange ofTrack:videoBTrack atTime:kCMTimeZero error:&error] )
            NSLog(@"%@", error);

        if ( ! [compositionTrackAudio insertTimeRange:_videoFullRange ofTrack:audioTrack atTime:kCMTimeZero error:&error] )
            NSLog(@"%@", error);

        _compositionTrackVideoA = [compositionTrackVideoA copy];
        _compositionTrackVideoB = [compositionTrackVideoB copy];

        AVPlayerItem * playerItem = [AVPlayerItem playerItemWithAsset:composition];

        AVPlayer * player = [AVPlayer playerWithPlayerItem:playerItem];

        [(AVPlayerLayer *)self.layer setPlayer:player];

        [player play];

        [player addObserver:self forKeyPath:@"status" options:0 context:0];

        [self updateCompositionForPlayerItem:playerItem];

        UITapGestureRecognizer * tapGesture = [[UITapGestureRecognizer alloc] initWithTarget:self action:@selector(didTap:)];
        [self addGestureRecognizer:tapGesture];
    }
    return self;
}

- (void)observeValueForKeyPath:(NSString *)keyPath ofObject:(id)object change:(NSDictionary *)change context:(void *)context
{
    if ( [keyPath isEqualToString:@"status"] )
        NSLog(@"STATUS %d", ((AVPlayer *)object).status );
}

- (void)updateCompositionForPlayerItem:(AVPlayerItem *)playerItem
{

    AVMutableVideoComposition * videoComposition = [AVMutableVideoComposition videoComposition];

    AVMutableVideoCompositionInstruction *videoInstruction = [AVMutableVideoCompositionInstruction videoCompositionInstruction];
    videoInstruction.enablePostProcessing = NO;
    videoInstruction.timeRange = _videoFullRange;

    AVMutableVideoCompositionLayerInstruction * layerInstructionA = [AVMutableVideoCompositionLayerInstruction videoCompositionLayerInstructionWithAssetTrack:_compositionTrackVideoA];
    CGAffineTransform transformA = CGAffineTransformMakeScale(0.5, 0.5);
    [layerInstructionA setTransform:transformA atTime:kCMTimeZero];
    AVMutableVideoCompositionLayerInstruction * layerInstructionB = [AVMutableVideoCompositionLayerInstruction videoCompositionLayerInstructionWithAssetTrack:_compositionTrackVideoB];
    CGAffineTransform transformB = CGAffineTransformMakeScale(0.5, 0.5);
    static int i = 0;
    transformB = CGAffineTransformTranslate(transformB, (i++ % 2 == 0) ? _videoSize.width : 0, _videoSize.height);
    [layerInstructionB setTransform:transformB atTime:kCMTimeZero];

    videoInstruction.layerInstructions = [NSArray arrayWithObjects:layerInstructionA, layerInstructionB, nil];

    videoComposition.instructions = [NSArray arrayWithObject:videoInstruction];

    videoComposition.frameDuration = CMTimeMake(1, 30); // 30 fps
    videoComposition.renderSize = _videoSize;

    playerItem.videoComposition = videoComposition;

}

- (void)didTap:(UITapGestureRecognizer *)tapGesture
{
    [self updateCompositionForPlayerItem:((AVPlayerLayer *)self.layer).player.currentItem];
}

@end

Answer 1

You can save the time where you want to change it and replace the player item with a new video composition and start the player again with the new player item from the time where you stoped playing.