FFmpeg libraries: Exactly constant segment duration for HLS

Question

We are using FFmpeg libraries git-ee94362 libavformat v55.2.100. Our purpose is to mux two streams (video and audio) into M3U8 playlist using HLS. In addition, we want the duration of every TS segment file be exactly 3.0 sec (frame rate is 25 fps).

To reach it, we are trying to set several options and properties, namely: - segment_time
- keyint_min - scenechange_threshold - gop_size - force_key_frames.

And our code looks as below:

AVCodecContext *codec_ctx = NULL;
AVFormatContext *ofmt_ctx = NULL;

int ret = 0, gopSize = (int)(3.0 * 25);   // 3 sec * 25 fps

// ofmt_ctx and codec_ctx initialization and filling are OK, but: 
codec_ctx->time_base.num = 1;
codec_ctx->time_base.den = 25 // fps

// It seems, that the following three lines have no effect without explisit setting of the "hls_time" property
codec_ctx->keyint_min = gopSize;       // in FFMpeg application, the corresponding option is "-keyint_min 3"
codec_ctx->scenechange_threshold = 0;  // in FFMpeg application, the corresponding option is "-sc_threshold 0"
codec_ctx->gop_size = gopSize;         // in FFMpeg application, the corresponding option is "-g 3"

ret = av_opt_set_double(ofmt_ctx, "hls_time", 3.0, AV_OPT_SEARCH_CHILDREN);

// Any of the following lines causes "Option not found" error.
ret = av_opt_set(codec_ctx->priv_data, "profile", "main", AV_OPT_SEARCH_CHILDREN);
ret = av_opt_set(codec_ctx->priv_data, "preset", "ultrafast", AV_OPT_SEARCH_CHILDREN);
ret = av_opt_get(ofmt_ctx, "segment_time",  AV_OPT_SEARCH_CHILDREN, &str);
ret = av_opt_set((ofmt_ctx, "segment_time", "3.0", AV_OPT_SEARCH_CHILDREN);

Anyway, the TS files durations are different, (~2-3 sec), and not EXACTLY 3.0 sec. Our question: What is the best way to solve the problem?

Andrey Mochenov.

Answer 1

The main problem you are facing is probably that your video file does not have key frames at fitting positions. That is especially a problem if you just copy the streams from the input.

FFmpeg is depending on key frames to calculate when to "cut" a segment. Makes sense when you think about it. You can't just make a cut between two key frames, as each segment needs to be fully functional on its own. Now, one could argue that FFmpeg should just insert new key frames on its own, then, but that would be too use friendly, wouldn't it ;)

Thankfully, you can force key frames using FFmpeg. Either using a parameter or setting the flag yourself in code. You said that you already tried forcing key frames, but I assume that you did not do it correctly.

This test of mine produces quite good results. It is command line only, sorry, but you already seem to know how to apply command line params in code, so you should be fine. Also note that I do not use the "hls_XXX" parameters because a) I honestly don't trust them and b) that way I figure it should also work for non-HLS streams.

ffmpeg -i inputFile.mov -force_key_frames "expr:gte(t,n_forced*10)" -strict -2 -c:a aac -c:v libx264 -f segment -segment_list_type m3u8 -segment_list_size 0 -segment_time 10.0 -segment_time_delta 0.1 -segment_list stream/test.m3u8 stream/test%02d.ts 

You can look up how exactly the force_key_frames command works here.

By now I implemented the above command in C++, with some additions. But without the "force_key_frames", as I set key frames manually during the transcoding process. Here is what I did:

AVDictionary* headerOptions(0);
av_dict_set(&headerOptions, "segment_format", "mpegts", 0);
av_dict_set(&headerOptions, "segment_list_type", "m3u8", 0);
av_dict_set(&headerOptions, "segment_list", _playlistFileName.c_str(), 0);
av_dict_set_int(&headerOptions, "segment_list_size", 0, 0);
av_dict_set(&headerOptions, "segment_time_delta", TO_STRING(1.00).c_str(), 0);
av_dict_set(&headerOptions, "segment_time", TO_STRING(_segmentDuration).c_str(), 0);
av_dict_set_int(&headerOptions, "reference_stream", _videoStream->index, 0);
av_dict_set(&headerOptions, "segment_list_flags", "cache+live", 0);
avformat_write_header(_formatContext, &headerOptions);

And here is the resulting m3u8:

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-ALLOW-CACHE:YES
#EXT-X-TARGETDURATION:11
#EXTINF:10.083333,
test00.ts
#EXTINF:10.000000,
test01.ts
#EXTINF:10.000000,
test02.ts
#EXTINF:10.000000,
test03.ts
#EXTINF:10.000000,
test04.ts
#EXTINF:10.000000,
test05.ts
#EXTINF:0.083333,
test06.ts
#EXT-X-ENDLIST

It is not perfect (first part is a little bit off somehow), but I am convinced you won't get better results than that.

Of course, the best option would be to make sure your input files always have correct key frames when just copying the streams, but sometimes you have no control over what files you get.

Side Note

When you are using FFmpeg in code, always try what you are doing in code using the cli ffmpeg command first. If you can get it to work that way, you at least know what parameters to set in code. And if it works using the command line tool, you know it must be possible in code somehow ;)