What does 'Past duration X.XXX too large' mean?

Question

One of the maintainers for the DVDStyler project on SourceForge said this about it:

FFMpeg versions after Jan 15 2015 often display this warning. It has been added to warn about possible rate control distortion, otherwise it does not cause any harm.

This warning message appears when trying to encode a high frame rate source to a low frame rate output, which means frames need to be dropped.

---

I had this error because I wanted to convert a series of images to a video:

ffmpeg -i %05d.png -r 24 -c:v libx264 -crf 5 out.mkv

The problem seems to be, that if no frame rate is give for the input, then a frame rate of 25 fps is assumed:

Input #0, image2, from 'frames/%04d.bmp':
  Duration: 00:00:15.96, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: bmp, bgra, 920x650, 25 fps, 25 tbr, 25 tbn, 25 tbc

This also can be seen on the total number of frames encoded. I had 400 images, but the above command only encoded 384:

frame=  384 fps= 68 q=-1.0 Lsize=   10931kB time=00:00:15.91 bitrate=5626.1kbits/s dup=0 drop=15    
video:10928kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.033807%

The error messages disappears by setting the input frame rate instead if the output frame rate. The output frame rate will then be automatically chosen to be that of the input. Additionally in newer ffmpeg versions you have to watch out, because when using PNG images with the -i option or rather the image2 or v4l2 input format, you have to use -framerate instead of -r, see the documentation for the -r option.

ffmpeg -framerate 24 -i %05d.png -c:v libx264 -crf 5 out.mkv

It is also possible to specify the frame rate of both input and output separately:

ffmpeg -framerate 25 -i %05d.png -r 10 -c:v libx264 -crf 5 out.mkv

In this case only 161/400 frames will be encoded. The other frames interim will be dropped. Also the error message disappears, I guess in order to not slow down ffmpeg by spamming to stdout, see:

• https://trac.ffmpeg.org/ticket/4700
• https://trac.ffmpeg.org/ticket/4401

Looking at the source code it seems to be that the difference between the presentation time (pts) in the input stream differs from the one in the output stream by more than a fixed limit set to 0.6 .

Snippets from the source:

    delta0 = sync_ipts - ost->sync_opts;
    delta  = delta0 + duration;

...

        if (delta0 < 0 &&
        delta > 0 &&
        format_video_sync != VSYNC_PASSTHROUGH &&
        format_video_sync != VSYNC_DROP) {
        double cor = FFMIN(-delta0, duration);
        if (delta0 < -0.6) {
            av_log(NULL, AV_LOG_WARNING, "Past duration %f too large\n", -delta0);
        } else
            av_log(NULL, AV_LOG_DEBUG, "Cliping frame in rate conversion by %f\n", -delta0);
        sync_ipts += cor;
        duration -= cor;
        delta0 += cor;
    }

This is only a quick glance, so please feel free to dig deeper.

I was getting thousands of these warnings with a particular encode. I was downscaling 1080p video to 480p. At an edit point, where there was some dodgy video due to a defect in the source laserdisc, these messages started coming up and then appeared for, I think, every frame thereafter. They went on and on, like this short excerpt:

Past duration 0.901115 too large=  535031kB time=00:54:15.06 bitrate=1346.5kbits/s dup=0 drop=19 speed=1.15x    
    Last message repeated 31 times
Past duration 0.901115 too large=  535031kB time=00:54:15.62 bitrate=1346.3kbits/s dup=0 drop=19 speed=1.15x    
    Last message repeated 34 times
Past duration 0.901115 too large=  535031kB time=00:54:16.21 bitrate=1346.0kbits/s dup=0 drop=19 speed=1.15x    
    Last message repeated 36 times
Past duration 0.901115 too large=  535338kB time=00:54:16.83 bitrate=1346.5kbits/s dup=0 drop=19 speed=1.15x    
    Last message repeated 39 times

The original ffmpeg invocation was this:

ffmpeg -i input.mp4 -s 720x480 -c:v libx264 -preset slower \
                              -crf 17 -c:a copy -y output.mkv

Following suggestions here I first added -framerate 60000/1001 to the input. That did not improve anything. I retained -framerate and added -r 60000/1001 to the output. That still did not improve anything. Retaining both I finally added -async 1 -vsync 1. This resulted in my receiving a single warning, and that's all. That invocation was:

ffmpeg -i input.mp4 -framerate 60000/1001 -s 720x480 -c:v libx264 \
           -preset slower -crf 17 -c:a copy -y output.mkv \
           -r 60000/1001 -async 1 -vsync 1

The only difference I found in a detailed dump from MediaInfo was the removal of this line found in the original invocation but not in the second one:

Delay relative to video                  : -33ms

However, I checked A/V sync near the beginning of the files and near the end, and there was no discernible difference in sync between the two files. Their running times were also the same, but that was only measured to the nearest second, in VLC. So I checked the frame counts using ffmpeg like so:

ffmpeg -i output.mkv -map 0:v:0 -c copy -f null -

and looking for "frame=#" near the end of the output.

Turns out the source video was 375226 frames long, the original invocation yielded 375195 frames, and the second invocation yielded 375200. So the second invocation, with vastly fewer warning messages also dropped 5 fewer frames.

Subsequent testing showed that -framerate and -r were unnecessary, and just using the two sync flags was sufficient. This produced identical results to the second invocation above, so the third and simplest invocation I found to solve the problem is this:

ffmpeg -i input.mp4 -s 720x480 -c:v libx264 -preset slower \
                   -crf 17 -c:a copy -y output.mkv -async 1 -vsync 1

And yet another file subsequently produced a bunch of these warnings even with the sync flags, but adding back the rate flags "fixed" it (only produced two instead of thousands of warnings). So sometimes the second invocation works when the third doesn't. For my immediate purposes I'm going to settle on the second invocation and hope it fixes most of these problems.

This was all with ffmpeg version 4.0.