How to sync multiple rtsp inputs in ffmpeg?

Question

I am combining multiple rtsp streams from local network cameras to a single output using ffmpeg. In the resulting video, the videos are out of sync.

ffmpeg -loglevel warning \
        -f image2 -loop 1 \
        -i logo.png \
        -rtsp_transport tcp -thread_queue_size 1024 \
        -i "rtsp://10.0.0.1" \
        -rtsp_transport tcp -thread_queue_size 1024 \
        -i "rtsp://10.0.0.2" \
        -rtsp_transport tcp -thread_queue_size 1024 \
        -i "rtsp://10.0.0.3" \
        -filter_complex "[3:v][2:v][0:v][1:v]xstack=inputs=4:layout=0_0|w0_0|0_h0|w0_h0[v]" \
        -map "[v]" \
        -filter_complex "[1:a][1:a]amerge=inputs=2[aout]" -map "[aout]" \
        -vcodec libx264 -pix_fmt yuv420p -r 30 -g 60 \
        -preset fast -b:v 2400k -maxrate 4600k -bufsize 6000k \
        -acodec aac -ar 44100 -b:a 128k \
        -f flv "rtmp://live-api-s.facebook.com:80/rtmp/$STREAMKEY"

If we take the first input (0) (10.0.0.1) as reference, the second input (1) is initially about 1000ms ahead, the third input (2) is about 2000ms ahead. It looks like the offset is even slightly increasing over time.

I tried to exchange the different inputs (e.g. 10.0.0.3 to input 0, 10.0.0.1 to input 2), then 10.0.0.1 is ahead of 10.0.0.3. So I would assume this is a matter of ffmpeg and how it processes the inputs.

How could I sync the different sources in the output?

Answer 1

Try adding -fflags nobuffer -flags low_delay -strict experimental before -i. you may see few currupted frames at the beginning since there is no reference frame.