Issue with AVAssetWriter Input: MPEG4

Question

I am compressing videos with AVAssetWriter. If i set the video compression file to Quicktime movie it works fine, however i would like to export it to MPEG4, but it gives me this error while running:

In order to perform passthrough to file type public.mpeg-4, please provide a format hint in the AVAssetWriterInput initializer'

Here is the specific code where i declare the file type:

 let videoInputQueue = DispatchQueue(label: "videoQueue")
    let audioInputQueue = DispatchQueue(label: "audioQueue")
    let formatter = DateFormatter()
    formatter.dateFormat = "yyyy'-'MM'-'dd'T'HH':'mm':'ss'Z'"
    let date = Date()
    let documentsPath = NSTemporaryDirectory()
    let outputPath = "\(documentsPath)/\(formatter.string(from: date)).mp4"
    let newOutputUrl = URL(fileURLWithPath: outputPath)

    do{
        assetWriter = try AVAssetWriter(outputURL: newOutputUrl, fileType: AVFileTypeMPEG4)
    }catch{
        assetWriter = nil
    }
    guard let writer = assetWriter else{
        fatalError("assetWriter was nil")
    }

    writer.shouldOptimizeForNetworkUse = true
    writer.add(videoInput)
    writer.add(audioInput)

Here is the full code for my compression:

func compressFile(urlToCompress: URL, completion:@escaping (URL)->Void){
    //video file to make the asset

    var audioFinished = false
    var videoFinished = false



    let asset = AVAsset(url: urlToCompress)

    //create asset reader
    do{
        assetReader = try AVAssetReader(asset: asset)
    } catch{
        assetReader = nil
    }

    guard let reader = assetReader else{
        fatalError("Could not initalize asset reader probably failed its try catch")
    }

    let videoTrack = asset.tracks(withMediaType: AVMediaTypeVideo).first!
    let audioTrack = asset.tracks(withMediaType: AVMediaTypeAudio).first!

    let videoReaderSettings: [String:Any] =  [kCVPixelBufferPixelFormatTypeKey as String!:kCVPixelFormatType_32ARGB ]

    // ADJUST BIT RATE OF VIDEO HERE

    let videoSettings:[String:Any] = [
        AVVideoCompressionPropertiesKey: [AVVideoAverageBitRateKey:self.bitrate],
        AVVideoCodecKey: AVVideoCodecH264,
        AVVideoHeightKey: videoTrack.naturalSize.height,
        AVVideoWidthKey: videoTrack.naturalSize.width
    ]


    let assetReaderVideoOutput = AVAssetReaderTrackOutput(track: videoTrack, outputSettings: videoReaderSettings)
    let assetReaderAudioOutput = AVAssetReaderTrackOutput(track: audioTrack, outputSettings: nil)


    if reader.canAdd(assetReaderVideoOutput){
        reader.add(assetReaderVideoOutput)
    }else{
        fatalError("Couldn't add video output reader")
    }

    if reader.canAdd(assetReaderAudioOutput){
        reader.add(assetReaderAudioOutput)
    }else{
        fatalError("Couldn't add audio output reader")
    }

    let audioInput = AVAssetWriterInput(mediaType: AVMediaTypeAudio, outputSettings: nil)
    let videoInput = AVAssetWriterInput(mediaType: AVMediaTypeVideo, outputSettings: videoSettings)
    videoInput.transform = videoTrack.preferredTransform
    //we need to add samples to the video input

    let videoInputQueue = DispatchQueue(label: "videoQueue")
    let audioInputQueue = DispatchQueue(label: "audioQueue")
    let formatter = DateFormatter()
    formatter.dateFormat = "yyyy'-'MM'-'dd'T'HH':'mm':'ss'Z'"
    let date = Date()
    let documentsPath = NSTemporaryDirectory()
    let outputPath = "\(documentsPath)/\(formatter.string(from: date)).mp4"
    let newOutputUrl = URL(fileURLWithPath: outputPath)

    do{
        assetWriter = try AVAssetWriter(outputURL: newOutputUrl, fileType: AVFileTypeMPEG4)
    }catch{
        assetWriter = nil
    }
    guard let writer = assetWriter else{
        fatalError("assetWriter was nil")
    }

    writer.shouldOptimizeForNetworkUse = true
    writer.add(videoInput)
    writer.add(audioInput)


    writer.startWriting()
    reader.startReading()
    writer.startSession(atSourceTime: kCMTimeZero)


    let closeWriter:()->Void = {
        if (audioFinished && videoFinished){
            self.assetWriter?.finishWriting(completionHandler: {

                self.checkFileSize(sizeUrl: (self.assetWriter?.outputURL)!, message: "The file size of the compressed file is: ")
                completion((self.assetWriter?.outputURL)!)
                print("Completed 1")

            })

            self.assetReader?.cancelReading()

        }
    }


    audioInput.requestMediaDataWhenReady(on: audioInputQueue) {
        while(audioInput.isReadyForMoreMediaData){
            let sample = assetReaderAudioOutput.copyNextSampleBuffer()
            if (sample != nil){
                audioInput.append(sample!)
            }else{
                audioInput.markAsFinished()
                DispatchQueue.main.async {
                    audioFinished = true
                    closeWriter()
                    print("Completed 2")
                }
                break;
            }
        }
    }

    videoInput.requestMediaDataWhenReady(on: videoInputQueue) {
        //request data here

        while(videoInput.isReadyForMoreMediaData){
            let sample = assetReaderVideoOutput.copyNextSampleBuffer()
            if (sample != nil){
                videoInput.append(sample!)
            }else{
                videoInput.markAsFinished()
                DispatchQueue.main.async {
                    videoFinished = true
                    print("Completed 3")
                    closeWriter()
                }
                break;
            }
        }

    }


}

Answer 1

By creating your audio AVAssetWriterInput with nil outputSettings you indicate that you want to pass through your audio data. The assetWriterInputWithMediaType:outputSettings: header file comment says:

AVAssetWriter only supports passing through a restricted set of media types and subtypes. In order to pass through media data to files other than AVFileTypeQuickTimeMovie, a non-NULL format hint must be provided using +assetWriterInputWithMediaType:outputSettings:sourceFormatHint: instead of this method.

A format description is needed and luckily you can get one from the sample buffers you encounter:

let formatDesc = CMSampleBufferGetFormatDescription(anAudioSampleBuffer)!   
let audioInput = AVAssetWriterInput(mediaType: AVMediaTypeAudio, outputSettings: nil, sourceFormatHint: formatDesc)

So if it's that easy to do, why doesn't AVAssetWriter do it for us? I guess because it would awkwardly push AVAssetWriter's usual initialization until some point after you've appended a few CMSampleBuffers, or (maybe?) because not all CMSampleBuffers have a format description.

Answer 2

I ran into this problem when trying to pass a .mp4 into an AVAssetWriter.

Specifically:

AVAssetReaderTrackOutput(track:,outputSettings:) outputSettings parameter to nil

and

AVAssetWriterInput(mediaType:,outputSettings:) outputSettings parameter to nil

Using this code from samkirkiles or for more a detailed look.

The problems are on lines 38 and 53:

// outputSettings: nil is the problem
let assetReaderAudioOutput = AVAssetReaderTrackOutput(track: audioTrack, outputSettings: nil)
let audioInput = AVAssetWriterInput(mediaType: AVMediaType.audio, outputSettings: nil)

Using a .mov that line works fine but using a .mp4 the nil value in the outputSettings parameter causes a crash: outputSettings: nil

To fix it add a dictionary with some audioSettings into the outputSettings parameter:

This is for line 38:

let audioOutputSettingsDict: [String : Any] = [
            AVFormatIDKey: kAudioFormatLinearPCM,
            AVSampleRateKey: 44100
]

let assetReaderAudioOutput = AVAssetReaderTrackOutput(track: audioTrack, outputSettings: audioOutputSettingsDict)

This is for line 53:

let audioInputSettingsDict: [String:Any] = [AVFormatIDKey : kAudioFormatMPEG4AAC,
                                   AVNumberOfChannelsKey : 2,
                                   AVSampleRateKey : 44100.0,
                                   AVEncoderBitRateKey: 128000 // he uses 250000 in his code via self.bitRate
]

let audioInput = AVAssetWriterInput(mediaType: AVMediaType.audio, outputSettings: audioInputSettingsDict)