ffmpeg failed to call avcodec_send_packet

Question

Windows 10 x64, ffmpeg: 3.1, prebuilt dlls

Absolute beginner here.

I was trying out a very basic case to decode frames from a video file but desperately failed at every attempts. The final code snippet looks like this:

#define __STDC_CONSTANT_MACROS
extern "C" {
#include<libavutil/avutil.h>
#include<libavutil/imgutils.h>
#include <libavformat/avformat.h>
#include <libavcodec/avcodec.h>
#include <libswscale/swscale.h>
}

#include <iostream>


int main(int argc, char **argv) {
    AVFormatContext* ctx_format = nullptr;
    AVCodecContext* ctx_codec = nullptr;
    AVCodec* codec = nullptr;
    AVFrame* frame = av_frame_alloc();
    int stream_idx;
    SwsContext* ctx_sws = nullptr;
    const char* fin = argv[1];
    AVStream *vid_stream = nullptr;
    AVPacket* pkt = av_packet_alloc();

    av_register_all();

    if (int ret = avformat_open_input(&ctx_format, fin, nullptr, nullptr) != 0) {
        std::cout << 1 << std::endl;
        return ret;
    }
    if (avformat_find_stream_info(ctx_format, nullptr) < 0) {
        std::cout << 2 << std::endl;
        return -1; // Couldn't find stream information
    }
    av_dump_format(ctx_format, 0, fin, false);

    for (int i = 0; i < ctx_format->nb_streams; i++)
        if (ctx_format->streams[i]->codecpar->codec_type == AVMEDIA_TYPE_VIDEO) {
            stream_idx = i;
            vid_stream = ctx_format->streams[i];
            break;
    }
    if (vid_stream == nullptr) {
        std::cout << 4 << std::endl;
        return -1;
    }

    codec = avcodec_find_decoder(vid_stream->codecpar->codec_id);
    if (!codec) {
        fprintf(stderr, "codec not found\n");
        exit(1);
    }
    ctx_codec = avcodec_alloc_context3(codec);

    if(avcodec_parameters_to_context(ctx_codec, vid_stream->codecpar)<0)
        std::cout << 512;
    if (avcodec_open2(ctx_codec, codec, nullptr)<0) {
        std::cout << 5;
        return -1;
    }

    //av_new_packet(pkt, pic_size);

    while(av_read_frame(ctx_format, pkt) >= 0){
        int ret = avcodec_send_packet(ctx_codec, pkt);
        if (ret < 0 || ret == AVERROR(EAGAIN) || ret == AVERROR_EOF) {
            std::cout << "avcodec_send_packet: " << ret << std::endl;
            break;
        }
        if (pkt->stream_index == stream_idx) {
            while (ret  >= 0) {
                ret = avcodec_receive_frame(ctx_codec, frame);
                if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF) {
                    //std::cout << "avcodec_receive_frame: " << ret << std::endl;
                    break;
                }
                std::cout << "frame: " << ctx_codec->frame_number << std::endl;
            }
        }
        av_packet_unref(pkt);
    }

    avformat_close_input(&ctx_format);
    av_packet_unref(pkt);
    avcodec_free_context(&ctx_codec);
    avformat_free_context(ctx_format);
    return 0;
}

It basically leaves everything by its defaults and try to read packets and decode frames from them. The avcodec_send_packet always returned a negative value at the first or second round of calling in the while loop.

Output for flv:

> .\streamer_test.exe .\SampleVideo_360x240_5mb.flv
Input #0, flv, from '.\SampleVideo_360x240_5mb.flv':
  Metadata:
    encoder         : Lavf53.24.2
  Duration: 00:00:53.32, start: 0.000000, bitrate: 787 kb/s
    Stream #0:0: Audio: aac (LC), 48000 Hz, 5.1, fltp, 384 kb/s
    Stream #0:1: Video: flv1, yuv420p, 320x240, 500 kb/s, 1k fps, 25 tbr, 1k tbn
frame: 1
[flv @ 000001545edb66c0] Bad picture start code
[flv @ 000001545edb66c0] header damaged
avcodec_send_packet: -1094995529

Output for mp4:

> .\streamer_test.exe .\SampleVideo_360x240_10mb.mp4
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '.\SampleVideo_360x240_10mb.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    creation_time   : 1970-01-01T00:00:00.000000Z
    encoder         : Lavf53.24.2
  Duration: 00:02:05.95, start: 0.000000, bitrate: 669 kb/s
    Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p, 320x240 [SAR 1:1 DAR 4:3], 282 kb/s, 15 fps, 15 t
br, 15360 tbn, 30 tbc (default)
    Metadata:
      creation_time   : 1970-01-01T00:00:00.000000Z
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, 5.1, fltp, 383 kb/s (default)
    Metadata:
      creation_time   : 1970-01-01T00:00:00.000000Z
      handler_name    : SoundHandler
[h264 @ 000002e2446a6cc0] Invalid NAL unit size (17191968 > 1007).
[h264 @ 000002e2446a6cc0] Error splitting the input into NAL units.
avcodec_send_packet: -1094995529

Output for mkv:

> .\streamer_test.exe .\SampleVideo_1280x720_2mb.mkv
Input #0, matroska,webm, from '.\SampleVideo_1280x720_2mb.mkv':
  Metadata:
    ENCODER         : Lavf53.24.2
  Duration: 00:00:10.65, bitrate: 1575 kb/s
    Stream #0:0: Video: mpeg4 (Simple Profile), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 1k tbn, 25 tbc (de
fault)
    Stream #0:1: Audio: aac (LC), 48000 Hz, 5.1, fltp (default)
frame: 1
[mpeg4 @ 000001166798ede0] header damaged
avcodec_send_packet: -1

So I guess there must be something wrong around avcodec_send_packet call but could not figured out what is missing. Any ideas about the causes? Much appreciate your help!

Answer 1

The call av_read_frame will return audio/video streams, you need to handle them with different AVCodecContext.

Because you send audio stream with avcodec_send_packet, but your AVCodecContext is for video stream, you got the error.

It seems you are only interested in video stream.

Change your code from

while(av_read_frame(ctx_format, pkt) >= 0){
    int ret = avcodec_send_packet(ctx_codec, pkt);
    if (ret < 0 || ret == AVERROR(EAGAIN) || ret == AVERROR_EOF) {
        std::cout << "avcodec_send_packet: " << ret << std::endl;
        break;
    }
    if (pkt->stream_index == stream_idx) {
        while (ret  >= 0) {
            ret = avcodec_receive_frame(ctx_codec, frame);
            if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF) {
                //std::cout << "avcodec_receive_frame: " << ret << std::endl;
                break;
            }
            std::cout << "frame: " << ctx_codec->frame_number << std::endl;
        }
    }
    av_packet_unref(pkt);
}

to the following will work.

while(av_read_frame(ctx_format, pkt) >= 0){
    if (pkt->stream_index == stream_idx) {
        int ret = avcodec_send_packet(ctx_codec, pkt);
        if (ret < 0 || ret == AVERROR(EAGAIN) || ret == AVERROR_EOF) {
            std::cout << "avcodec_send_packet: " << ret << std::endl;
            break;
        }
        while (ret  >= 0) {
            ret = avcodec_receive_frame(ctx_codec, frame);
            if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF) {
                //std::cout << "avcodec_receive_frame: " << ret << std::endl;
                break;
            }
            std::cout << "frame: " << ctx_codec->frame_number << std::endl;
        }
    }
    av_packet_unref(pkt);
}

When decoding use ffmpeg, remember to call avcodec_send_packet/avcodec_receive_frame in pair for different streams.