Record video with AVAssetWriter: first frames are black

Question

I am recording video (the user also can switch to audio only) with AVAssetWriter. I start the recording when the app is launched. But the first frames are black (or very dark). This also happens when I switch from audio to video. It feels like the AVAssetWriter and/or AVAssetWriterInput are not yet ready to record. How can I avoid this?

I don't know if this is a useful info but I also use a GLKView to display the video.

func start_new_record(){
    do{
        try self.file_writer=AVAssetWriter(url: self.file_url!, fileType: AVFileTypeMPEG4)
        if video_on{
            if file_writer.canAdd(video_writer){
                file_writer.add(video_writer)
            }
        }
        if file_writer.canAdd(audio_writer){
            file_writer.add(audio_writer)
        }
    }catch let e as NSError{
        print(e)
    }
}

func captureOutput(_ captureOutput: AVCaptureOutput!, didOutputSampleBuffer sampleBuffer: CMSampleBuffer!, from connection: AVCaptureConnection!){
    guard is_recording else{
        return
    }

    guard CMSampleBufferDataIsReady(sampleBuffer) else{
        print("data not ready")
        return
    }

    guard let w=file_writer else{
        print("video writer nil")
        return
    }

    if w.status == .unknown && start_recording_time==nil{
        if (video_on && captureOutput==video_output) || (!video_on && captureOutput==audio_output){
            print("START RECORDING")
            file_writer?.startWriting()
            start_recording_time=CMSampleBufferGetPresentationTimeStamp(sampleBuffer)
            file_writer?.startSession(atSourceTime: start_recording_time!)
        }else{
            return
        }
    }

    if w.status == .failed{
        print("failed /", w.error ?? "")
        return
    }

    if captureOutput==audio_output{
        if audio_writer.isReadyForMoreMediaData{
            if !video_on || (video_on && video_written){
                audio_writer.append(sampleBuffer)
                //print("write audio")
            }
        }else{
            print("audio writer not ready")
        }
    }else if video_output != nil && captureOutput==video_output{
        if video_writer.isReadyForMoreMediaData{
            video_writer.append(sampleBuffer)
            if !video_written{
                print("added 1st video frame")
                video_written=true
            }
        }else{
            print("video writer not ready")
        }
    }
}

Answer 1

SWIFT 4

SOLUTION #1:

I resolved this by calling file_writer?.startWriting() as soon as possible upon launching the app. Then when you want to start recording, do the file_writer?.startSession(atSourceTime:...).

When you are done recording and call finishRecording, when you get the callback that says that's complete, set up a new writing session again.

SOLUTION #2:

I resolved this by adding half a second to the starting time when calling AVAssetWriter.startSession, like this:

start_recording_time = CMSampleBufferGetPresentationTimeStamp(sampleBuffer)
let startingTimeDelay = CMTimeMakeWithSeconds(0.5, 1000000000)
let startTimeToUse = CMTimeAdd(start_recording_time!, startingTimeDelay)

file_writer?.startSession(atSourceTime: startTimeToUse)

SOLUTION #3:

A better solution here is to record the timestamp of the first frame you receive and decide to write, and then start your session with that. Then you don't need any delay:

//Initialization, elsewhere:

var is_session_started = false
var videoStartingTimestamp = CMTime.invalid

// In code where you receive frames that you plan to write:

if (!is_session_started) {
    // Start writing at the timestamp of our earliest sample
    videoStartingTimestamp = currentTimestamp
    print ("First video sample received: Starting avAssetWriter Session: \(videoStartingTimestamp)")
    avAssetWriter?.startSession(atSourceTime: videoStartingTimestamp)

    is_session_started = true
}

// add the current frame
pixelBufferAdapter?.append(myPixelBuffer, withPresentationTime: currentTimestamp)

Answer 2

Ok, stupid mistake...

When launching the app, I init my AVCaptureSession, add inputs, outputs, etc. And I was just calling start_new_record a bit too soon, just before commitConfiguration was called on my capture session.

At least my code might be useful to some people.