How to trigger redraw of filter in AVVideoComposition's applyingCIFiltersWithHandler?

Question

I'm using Swift to show content from an AVPlayer in a view's AVPlayerLayer. The associated AVPlayerItem has a videoComposition, and slightly simplified version of the code to create it (without error checking, etc.) looks like this:

playerItem.videoComposition = AVVideoComposition(asset: someAsset, applyingCIFiltersWithHandler: {
                [unowned self] (request: AVAsynchronousCIImageFilteringRequest) in

    let paramDict = << set up parameter dictionary based on class vars >>

    // filter the image
    let filter = self.ciFilterWithParamDict(paramDict) {
    filter.setValue(request.sourceImage, forKey: kCIInputImageKey)
    if let filteredImage = filter.outputImage {
        request.finishWithImage(filteredImage, context: nil)
    }
})

This all works as expected when the AVPlayer is playing or seeking. And if a new videoComposition is created and loaded, the AVPlayerLayer is rendered correctly.

I have not found a way, however, to "trigger" the AVPlayer/ AVPlayerItem/ AVVideoComposition to re-render when I have changed some of the values that I use to calculate filter parameters. If I change values and then play or seek, it is rendered correctly, but only if I play or seek. Is there no way to trigger a rendering "in place"?

Answer 1

The best way that I know to do this is to simply create a new AVVideoComposition instance for the AVPlayerItem when editing your CIFilter inputs on a paused AVPlayer. In my experience it's way faster and cleaner than swapping player items out and back into the player. You might think that creating a new video composition is slow, but really all you are doing is redefining the render path at that specific frame, which is almost as efficient as invalidating the part of the Core Image cache that was impacted by your change.

The key here that the video composition of the player item must be invalidated in some way to trigger a redraw. Simply changing the input parameters of the Core Image filters have sadly no way (that i know of) of invalidating the video composition, which is the source of the issue.

You can get even more efficient by creating a AVMutableVideoComposition instance for the AVPlayerItem and mutate it in some ways (by changing things like instructions, animationTool, frameDuration) when editing on pause.

Answer 2

I used a hack to replace the avPlayerItem entirely to force a refresh. But it would be much better if there was a way to trigger the avPlayerItem to re-render directly.

// If the video is paused, force the player to re-render the frame.
if (self.avPlayer.currentItem.asset && self.avPlayer.rate == 0) {
    CMTime currentTime = self.avPlayerItem.currentTime;

    [self.avPlayer replaceCurrentItemWithPlayerItem:nil];
    [self.avPlayer replaceCurrentItemWithPlayerItem:self.avPlayerItem];
    [self.avPlayerItem seekToTime:currentTime toleranceBefore:kCMTimeZero toleranceAfter:kCMTimeZero];
}

Answer 3

Along the lines of this answer, but instead of creating a brand new AVVideoComposition and setting that as the player's videoComposition, it appears you can force a refresh of the current frame by simply setting videoComposition to nil and then immediately back to the existing videoComposition instance.

This results in the following simple workaround any time you want to force refresh the current frame:

let videoComposition = player.currentItem?.videoComposition
player.currentItem?.videoComposition = nil
player.currentItem?.videoComposition = videoComposition

Answer 4

This one might help you.

let currentTime = self.player.current()
self.player.play()
self.player.pause()
self.player.seek(to: currentTime, toleranceBefore:kCMTimeZero toleranceAfter:kCMTimeZero)