Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Setting AVMutableVideoComposition instruction causes handler not to be called

I have a function to filter an AVPlayerItem's asset. One of the problems was setting the transform of the video. However, whenever I set the AVMutableVideoCompositionInstruction of the AVMutableVideoComposition, the handler is no longer called.

Here is my code:

private func filter(playerItem: AVPlayerItem) {

    let videoComposition = AVMutableVideoComposition(asset: playerItem.asset, applyingCIFiltersWithHandler: { (request) in
        print("Composing") // does not print whenever the instructions are added
        if let filteredImage = filterImage(request.sourceImage) {
            request.finish(with: filteredImage, context: nil)
        } else {
            request.finish(with: RenderError.couldNotFilter) // An error
        }
    })

    guard let videoTrack = playerItem.asset.tracks(withMediaType: .video).first else { return }

    let size = CGSize(width: videoTrack.naturalSize.height, height: videoTrack.naturalSize.width)
    videoComposition.renderSize = size

    let videoInstruction = AVMutableVideoCompositionInstruction()
    videoInstruction.timeRange = CMTimeRange(start: kCMTimeZero, duration: playerItem.asset.duration)

    let transformInstruction = AVMutableVideoCompositionLayerInstruction(assetTrack: videoTrack)
    let translate = CGAffineTransform(translationX: size.width, y: size.height)
    let rotate = CGAffineTransform(rotationAngle: CGFloat.pi)
    transformInstruction.setTransform(translate.concatenating(rotate), at: kCMTimeZero)
    videoInstruction.layerInstructions.append(transformInstruction)
    videoComposition.instructions.append(videoInstruction)

    playerItem.videoComposition = videoComposition
}

Why is the handler no longer called, and how can I fix it?

I will give you so many brownie points if you can answer!

like image 414
IHaveAQuestion Avatar asked Jul 13 '17 20:07

IHaveAQuestion


1 Answers

I filed a bug report with Apple, and apparently this behavior is not a bug. This was their response:

Engineering has provided the following information regarding this issue:

CoreImage filtering and layer instruction based composition can't be used simultaneously. Layer instructions won't be run when added to an AVMutableVideoComposition that it is initialized with +[videoCompositionWithAsset:applyingCIFiltersWithHandler:]. To use layer instructions in this case, move the functionality into the handler instead of adding the layer instructions to the AVMutableVideoComposition.

This explains why the instructions seem to have not been doing anything, and the handler not called. They say to move the transform functionality to the handler instead of using instructions; unfortunately, I do not quite know how to implement this solution –– that is another question.

like image 149
Daniel Avatar answered Nov 11 '22 18:11

Daniel