How to extract closedcaptions/subtitle from AVPlayer

Question

Since closed-captions rendered by AVPlayer sometimes overlap with other UI components I would like to render cc in a separate view.

I am able to turn off AVPlayer's cc rendering by setting closedCaptionDisplayEnabled to NO but I did not find a way to extract the closed captions I want to render.

Does anybody know if there's a way to extract CC string from AVPlayer/AVPlayerItem? I'm able to identify the AVAssetTrack with AVMediaTypeClosedCaption but I am not sure how to extract the string for a certain time.

Answer 1

The key steps to "extracting" the caption strings are

1. Create an output: let captionOutput = AVPlayerItemLegibleOutput().
2. Set ourselves as a delegate: captionOutput.setDelegate(self, queue: DispatchQueue.main).
3. When the stream is ready, add the output: player.currentItem?.add(captionOutput).
4. Create a delegate extension to get the caption changes:

extension ViewController: AVPlayerItemLegibleOutputPushDelegate {
    func legibleOutput(_ output: AVPlayerItemLegibleOutput,
                       didOutputAttributedStrings strings: [NSAttributedString],
                       nativeSampleBuffers nativeSamples: [Any],
                       forItemTime itemTime: CMTime) {
        // Your attributed caption strings get delivered here!
    }
}

5. Optionally, supress captions on the player: captionOutput.suppressesPlayerRendering = true.

I've created an example project here: https://github.com/balnaves/AVPlayerItemLegibleOutputTest