Multiple frames lost if I use av_read_frame in FFmpeg

Question

I have an HEVC sequence with 3500 frames and I am writing a decoder for reading it (read frame by frame and dump to yuv). In my main(), I have a for loop that calls a decoder() 3500 times (I am assuming at this stage that the main() knows how many frames there are).

So, for every call to decoder(), I need a complete frame to be returned. This is what the decoder() looks like..

bool decode(pFormatCtx, pCodecCtx)
{
    int gotaFrame=0;

    while (gotaFrame==0) {

        printf("1\t");

        if ( !av_read_frame(pFormatCtx, &packet) ) { 
            if(packet.stream_index==videoStreamIndex) {

                // try decoding
                avcodec_decode_video2(pCodecCtx, pFrame, &gotaFrame, &packet);

                if (gotaFrame) {  // decode success.

                    printf("2\t");

                    // dump to yuv ... not shown here. 

                    // cleanup
                    av_frame_unref(pFrame);
                    av_frame_free(&pFrame);
                    av_free_packet(&packet);

                    return true;
                }
            }
        }
    }
}

The behavior is like this: 1 1 1 1 1 1 1 1 1 1 1 1 1 2 1 2 1 2 ...... it looks like it reads several frames before decoding one? The first frame is an I-frame, so shouldn't that be decoded right away?

With this code, I end up losing the several frames (indicated by the series of 1s). Can someone help me out here? Is there something I am doing wrong in my code?

Update: the test clip is video-only. No audio.

Answer 1

What you are seeing is the correct behavior. The decoder bufferers a few frames for multithreaded efficiency. And it may take several frames to 'prime the pump' as it were. Basically, to keep your program responsive, avcodec_decode_video2 queues up the frame for decoding, then returns. This prevents your program from blocking for a long time. It is also absolutely required to delay decoding in the case of B frames where the decode order may not be the same as the display order.

So, how to not lose these frames? After av_read_frame stops returning new frames, you must flush the decoder by calling avcodec_decode_video2 with empty packets until no more frames are retuned.