How to read any frame while having frame number using ffmpeg av_seek_frame()

Question

int64_t timeBase;
timeBase = (int64_t(pavStrm-> time_base.num) * AV_TIME_BASE) / int64_t(pavStrm->time_base.den);
int64_t seekTarget = int64_t(iFrameNumber) * timeBase;
av_seek_frame(fmt_ctx, -1, seekTarget, AVSEEK_FLAG_FRAME);

here I want to read next 5 frame after iFrameNumebr

for(int iCnt = 0; iCnt <= 4; iCnt++)
{
    iRet = av_read_frame(fmt_ctx, &pkt);
        do 
        {
            ret = decode_packet(&got_frame, 0);
            if (ret < 0)
                break;
            pkt.data += ret;
            pkt.size -= ret;

        }while (pkt.size > 0);
    av_free_packet(&pkt);
}

static int decode_packet(int *got_frame, int cached)
{
int ret = 0;
int decoded = pkt.size;
*got_frame = 0;

if (pkt.stream_index == video_stream_idx)
{
    /* decode video frame */
    ret = avcodec_decode_video2(video_dec_ctx, frame, got_frame, &pkt);
}

when i am using AVSEEK_FLAG_BACKWARD its return 5 packet and 5 frame first two is blank but correct.

when i am using AVSEEK_FLAG_FRAME its return 5 packet and 3 frame which are not first 3 frame its return specific frame from video.

for any iFrameNumber

so please help me how to get frame while having frame number and what is exact value of seektarget 3rd param of av_seek_frame()

also I have problem while converting frame to rgb24 format

Answer 1

I think av_seek_frame() is one of the most common but difficult to understand function, also not well commented enough.

If the flag AVSEEK_FLAG_FRAME is set, the third parameter should be a frame number you want to seek, which you're doing fine.

Let's see a example to have a better understand of av_seek_frame():

Say I have a video of 10 frames, with fps=10. The first and fifth frame is key frame (I Frame or intra frame). Others are P frames or even B frames in some format.

0 1 2 3 4 5 6 7 8 9 (frame number)

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 (timebase)

av_seek_frame(fmt_ctx, -1, 2, AVSEEK_FLAG_FRAME);
av_seek_frame(fmt_ctx, -1, 0.15, 0);
// These will seek to the fifth frame. Cause `AVSEEK_FLAG_ANY` is not given. Seeking to the next key frame after third parameter.

av_seek_frame(fmt_ctx, -1, 2, AVSEEK_FLAG_FRAME | AVSEEK_FLAG_ANY);
// This will seek to exactly the third parameter specified. But probably only a frame with no actual meaning. (We can't get a meaningful image if no related I/P/B frames given.)

av_seek_frame(fmt_ctx, -1, 0.15, AVSEEK_FLAG_ANY);
// Seek to 0.2. Nothing interesting as above.

av_seek_frame(fmt_ctx, -1, 0.15, AVSEEK_FLAG_ANY | AVSEEK_FLAG_BACKWARD);
// Seek to 0.1. Also nothing interesting.

av_seek_frame(fmt_ctx, -1, 2, AVSEEK_FLAG_FRAME | AVSEEK_FLAG_BACKWARD);
// Got the first frame. Seeking to the nearest key frame before the third parameter.

So if I'd like to get arbitrary frame, usually seeking with AVSEEK_FLAG_BACKWARD first, decoding as usual. Then check the first several packets pts and duration, see if we need to drop them.

Answer 2

int64_t FrameToPts(AVStream* pavStream, int frame) const
{
return (int64_t(frame) * pavStream->r_frame_rate.den *  pavStream-
>time_base.den) / 
(int64_t(pavStream->r_frame_rate.num) * 
pavStream->time_base.num);
}

iSeekTarget = FrameToPts(m_pAVVideoStream, max(0, lFrame));
iSuccess = av_seek_frame(m_pAVFmtCtx, m_iVideo_Stream_idx, 
iSeekTarget, iSeekFlag);

AVPacket avPacket;
iRet = av_read_frame(m_pAVFmtCtx, &avPacket);