Insert frames into video using FFMPEG

Question

The camera I am recording video with occasionally drops frames, so I need to reinsert frames into the video to keep the timing precise.

I already have a script to identify exactly when those frames were dropped, so I have an index of every frame that needs to be inserted and where.

For example, in a 100 s video at 100 FPS, I should have 10,000 frames. However, 4 frames were dropped at frame 399, 1205, 4299, and 7891. So, I want to either insert a black frame at the same resolution at those spots, or hold the previous frame for exactly one frame (e.g. hold frame 398 for an extra frame, or 0.01 s).

Is there a way to do this iteratively in FFMPEG? I am currently writing the video into its constituent frames, adding in my blank images, then re-concatenating the video from the frames, which is a very inefficient process.

Answer 1

Let's take a video having frame rate value FR and your missing frames having index 399, 1205, 4299, and 7891. First frame has index 0.

ffmpeg -i in.avi -vf setpts='PTS+(1/FR/TB)*(gte(N,399)+gte(N,1205)+gte(N,4299)+gte(N,7891))' -vsync cfr -q:v 1 out.avi

The setpts filter can adjust timestamps and what the setpts expression does is offset all frames after the given indices forward. The amount offset is evaluated based on how many earlier frames are missing. This will create empty timestamp slots. The -vsync cfr option will then fill in these slots with a clone of the previous available frame.