Reading RTSP stream with FFMpeg library - how to use avcodec_open2?

Question

While trying to read rtsp stream I get some problems, with code and documentation alike. Short description: whatever I do, avcodec_open2 either fails (saying "codec type or id mismatches") or width and height of codec context after the call are 0 (thus making further code useless). Stream itself can be opened normally by VLC player and av_dump_format() displays correct info. My code is based on technique answer to this question.

Long description: my code is in C#, but here is C++-equivalent of FFMpeg calls (I actually reduced my code to this minimum and problem persists):

av_register_all();
avformat_network_init(); //return code ignored

AVFormatContext* formatContext = avformat_alloc_context();
if (avformat_open_input(&formatContext, stream_path, null, null) != 0) {
    return;
}

if (avformat_find_stream_info(formatContext, null) < 0) {
    return;
}

int videoStreamIndex = 0;
for (int i = 0; i < formatContext->nb_streams; ++i) {
    AVStream* s = formatContext->streams[i];
    if (s->codec == null) continue;
    AVCodecContext c = *(s->codec);
    if (c.codec_type == AVMEDIA_TYPE_VIDEO) videoStreamIndex = i;
}

//start reading packets from stream and write them to file
//av_read_play(formatContext); //return code ignored
//this call would print "method PLAY failed: 455 Method Not Valid in This State"
//seems to be the case that for rtsp stream it isn't needed

AVCodec* codec = null;
codec = avcodec_find_decoder(AV_CODEC_ID_H264);
if (codec == null) {
    return;
}

AVCodecContext* codecContext = avcodec_alloc_context3(null);
avcodec_get_context_defaults3(codecContext, codec);//return code ignored
avcodec_copy_context(codecContext, formatContext->streams[videoStreamIndex]->codec); //return code ignored

av_dump_format(formatContext, videoStreamIndex, stream_path, 0);

if (avcodec_open2(codecContext, codec, null) < 0) {
    return;
}

The code actually uses DLL version of FFMpeg library; avcodec-55.dll and avformat-55.dll are used.

Documentation says something weird about which calls can be made in which succession (that copy_context should be called before get_context_defaults), current code is left close as possible to technique version. As written, it results in non-zero return from avcodec_open2 with "codec type or id mismatches" message. Changing the order does little good: now avcodec_open2 executes successfully, but both codecContext->width and codecContext->height are 0 afterwards.

Also documentation doesn't mention which is default value for the third argument of avcodec_open2 should be, but source code seems to taking into account that options can be NULL.

Output of av_dump_format is as follows:

Input #0, rtsp, from 'rtsp://xx.xx.xx.xx:xx/video.pro1':
  Metadata:
    title           : QStream
    comment         : QStreaming Media
  Duration: N/A, start: 0.000000, bitrate: 64 kb/s
    Stream #0:0: Video: h264 (Baseline), yuvj420p(pc), 1920x1080, 30 fps, 25 tbr, 90k tbn, 60 tbc
    Stream #0:1: Audio: pcm_mulaw, 8000 Hz, 1 channels, s16, 64 kb/s

Answer 1

First, what does the av_dump_format shows? Are you sure your video stream codec is h264, because you try to open the codec as if it were H264.

In order to open any codec, change your avcodec_find_decoder to pass it the source codec id:

codec = avcodec_find_decoder(formatContext->streams[videoStreamIndex]->codec->codec_id);

By the way, (forget this one if you do not use the c++ code but stick with c#): you do not need to make a copy of the initial AVCodecContext when you are looking for the video stream. You can do: (note that you may want to keep a pointer to the inital codec context, see below).

AVCodecContext* c = s->codec;
if (c->codec_type == AVMEDIA_TYPE_VIDEO) {
    videoStreamIndex = i;
    initialVideoCodecCtx = c;
}

Next point, not really relevant in this case: instead of looping through all the steams, FFmpeg has a helper function for it:

int videoStreamIndex = av_find_best_stream(formatContext, AVMEDIA_TYPE_VIDEO, -1, -1, NULL, 0);

Last point: I think only the first point should do the trick to make avcodec_open2 work, but you might not be able to decode your stream. You opened the codec for the new codec context, but no codec is opened for the inital context. Why did you make a copy of the initial codec context? It is usefull if you want to record your stream in another file (i.e. transcode), but if you only want to decode your stream, it is much easier to use the initial context, and use it instead of the new one as a parameter for avcodec_decode_video2.

To sum it up, replace your code after avformat_find_stream_info by (warning: no error check):

int videoStreamIndex = av_find_best_stream(formatContext, AVMEDIA_TYPE_VIDEO, -1, -1, NULL, 0);
AVCodecContext* codecCtx = formatContext->streams[videoStreamIndex]->codec;
AVCodec* codec = avcodec_find_decoder(codecCtx->codec_id);
// tune codecCtx if you want special decoding options. See FFmpeg docs for a list of members
if (avcodec_open2(codecCtx, codec, null) < 0) {
    return;
}

// use av_read_frame(formatContext, ...) to read packets
// use avcodec_decode_video2(codecCtx, ...) to decode packets