FFMpeg Out of sync audio/video in iOS application

Question

The app saves the camera output into a mov. file, then turn it to flv format that sent by AVPacket to rtmp server. It switch every time between two files, one is written by the camera output and the other one is sent. My problem is that the audio/video is getting out of sync after a while.

The first buffer sent is allways 100% sync but after awhile it get messed. I belive its a DTS-PTS problem..

 if(isVideo)
{
    packet->stream_index = VIDEO_STREAM;
   packet->dts = packet->pts = videoPosition;
    videoPosition += packet->duration = FLV_TIMEBASE * packet->duration * videoCodec->ticks_per_frame * videoCodec->time_base.num / videoCodec->time_base.den;

}
else
{
    packet->stream_index = AUDIO_STREAM;
    packet->dts = packet->pts = audioPosition;
    audioPosition += packet->duration = FLV_TIMEBASE * packet->duration / audioRate;

   //NSLog(@"audio position = %lld", audioPosition);
}

packet->pos = -1;
packet->convergence_duration = AV_NOPTS_VALUE;

// This sometimes fails without being a critical error, so no exception is raised
if((code = av_interleaved_write_frame(file, packet)))
{
    NSLog(@"Streamer::Couldn't write frame");
}
av_free_packet(packet);

Answer 1

You can research this sample: http://unick-soft.ru/art/files/ffmpegEncoder-vs2008.zip

But this sample is for Windows.

In this sample I use pts only for audio stream:

  if (pVideoCodec->coded_frame->pts != AV_NOPTS_VALUE)
  {
    pkt.pts = av_rescale_q(pVideoCodec->coded_frame->pts, 
      pVideoCodec->time_base, pVideoStream->time_base);
  }

Answer 2

I was experiencing a similar issue when switching out the AVAssetWriters, and noticed that it went way if I only started using the new AVAssetWriter when I got a video sample

https://medium.com/@brandon.kobel/ios-seamless-video-chunks-4383a5a3a874