Data corruption when reading realtime H.264 output from AVAssetWriter

Question

I'm using some tricks to try to read the raw output of an AVAssetWriter while it is being written to disk. When I reassemble the individual files by concatenating them, the resulting file is the same exact number of bytes as the AVAssetWriter's output file. However, the reassembled file will not play in QuickTime or be parsed by FFmpeg because there is data corruption. A few bytes here and there have been changed, rendering the resulting file unusable. I assume this is occurring on the EOF boundary of each read, but it isn't consistent corruption.

I plan to eventually use code similar to this to parse out individual H.264 NAL units from the encoder to packetize them and send them over RTP, however if I can't trust the data being read from disk I might have to use another solution.

Is there an explanation/fix for this data corruption? And are there any other resources/links you have found on how to parse the NAL units to packetize over RTP?

Full code here: AVAppleEncoder.m

// Modified from
// http://www.davidhamrick.com/2011/10/13/Monitoring-Files-With-GCD-Being-Edited-With-A-Text-Editor.html
- (void)watchOutputFileHandle
{
    dispatch_queue_t queue = dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0);
    int fildes = open([[movieURL path] UTF8String], O_EVTONLY);

    source = dispatch_source_create(DISPATCH_SOURCE_TYPE_VNODE,fildes,
                                                              DISPATCH_VNODE_DELETE | DISPATCH_VNODE_WRITE | DISPATCH_VNODE_EXTEND | DISPATCH_VNODE_ATTRIB | DISPATCH_VNODE_LINK | DISPATCH_VNODE_RENAME | DISPATCH_VNODE_REVOKE,
                                                              queue);
    dispatch_source_set_event_handler(source, ^
                                      {
                                          unsigned long flags = dispatch_source_get_data(source);
                                          if(flags & DISPATCH_VNODE_DELETE)
                                          {
                                              dispatch_source_cancel(source);
                                              //[blockSelf watchStyleSheet:path];
                                          }
                                          if(flags & DISPATCH_VNODE_EXTEND)
                                          {
                                              //NSLog(@"File size changed");
                                              NSError *error = nil;
                                              NSFileHandle *fileHandle = [NSFileHandle fileHandleForReadingFromURL:movieURL error:&error];
                                              if (error) {
                                                  [self showError:error];
                                              }
                                              [fileHandle seekToFileOffset:fileOffset];
                                              NSData *newData = [fileHandle readDataToEndOfFile];
                                              if ([newData length] > 0) {
                                                  NSLog(@"newData (%lld): %d bytes", fileOffset, [newData length]);
                                                  NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
                                                  NSString *basePath = ([paths count] > 0) ? [paths objectAtIndex:0] : nil;
                                                  NSString *movieName = [NSString stringWithFormat:@"%d.%lld.%d.mp4", fileNumber, fileOffset, [newData length]];
                                                  NSString *path = [NSString stringWithFormat:@"%@/%@", basePath, movieName];
                                                  [newData writeToFile:path atomically:NO];
                                                  fileNumber++;
                                                  fileOffset = [fileHandle offsetInFile];
                                              }
                                          }
                                      });
    dispatch_source_set_cancel_handler(source, ^(void) 
                                       {
                                           close(fildes);
                                       });
    dispatch_resume(source);
}

Here are some similar questions I have found, but don't exactly answer my question:

• Get PTS from raw H264 mdat generated by iOS AVAssetWriter
• streaming video FROM an iPhone
• Parsing h.264 NAL units from a quicktime MOV file
• Realtime Audio/Video Streaming FROM iPhone to another device (Browser, or iPhone)

When I eventually figure this out, I will release an open source library to assist people who try to do this in the future.

Thank you!

Update: The corruption doesn't happen at the EOF boundary. It seems like parts of the file are re-written after finishWriting is called. This first file was chunked at 4KB, so the area changed isn't anywhere near an EOF boundary. It seems to be corrupted near new "moov" elements as well when movieFragmentInterval is enabled.

Correct file on the left, broken file on the right.

Answer 1

I ended up abandoning the "read while it's written" approach in favor of a manual chunking approach where I call finishWriting every 5 seconds on a background thread. I was able to drop a negligible number of frames using a method originally described here:

- (void) segmentRecording:(NSTimer*)timer {
    AVAssetWriter *tempAssetWriter = self.assetWriter;
    AVAssetWriterInput *tempAudioEncoder = self.audioEncoder;
    AVAssetWriterInput *tempVideoEncoder = self.videoEncoder;
    self.assetWriter = queuedAssetWriter;
    self.audioEncoder = queuedAudioEncoder;
    self.videoEncoder = queuedVideoEncoder;
    //NSLog(@"Switching encoders");

    dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0), ^{
        [tempAudioEncoder markAsFinished];
        [tempVideoEncoder markAsFinished];
        if (tempAssetWriter.status == AVAssetWriterStatusWriting) {
            if(![tempAssetWriter finishWriting]) {
                [self showError:[tempAssetWriter error]];
            }
        }
        if (self.readyToRecordAudio && self.readyToRecordVideo) {
            NSError *error = nil;
            self.queuedAssetWriter = [[AVAssetWriter alloc] initWithURL:[self newMovieURL] fileType:(NSString *)kUTTypeMPEG4 error:&error];
            if (error) {
                [self showError:error];
            }
            self.queuedVideoEncoder = [self setupVideoEncoderWithAssetWriter:self.queuedAssetWriter formatDescription:videoFormatDescription bitsPerSecond:videoBPS];
            self.queuedAudioEncoder = [self setupAudioEncoderWithAssetWriter:self.queuedAssetWriter formatDescription:audioFormatDescription bitsPerSecond:audioBPS];
            //NSLog(@"Encoder switch finished");

        }
    });
}

Full source code: https://github.com/chrisballinger/FFmpeg-iOS-Encoder/blob/master/AVSegmentingAppleEncoder.m

Answer 2

When reading a MOV file that is ACTIVELY recording on iOS, you MUST check the 4 bytes mentioned for changes, and re-write this four bytes, then check for additional data in file, and send additional data. Then when done, truncate the file to the file size written.

Obviously this depends on where you are sending the file. I use a send (offset,number of bytes) to receiver. So I send "additional data", "more additional data", ... , new data at (24,4), "more additional data".

Typically iOS only writes the 4 byte (size of data section) record when file is about to be closed (aka after last media write). (see info on "Quicktime atoms"). Unfortunately, this also means the MOV file is not PLAYABLE until recording is completed (and movie descriptors written at END of file).