Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

streaming an mkv file while processing with ffmpeg

What I want to do:

  • I want to play an mkv video in Firefox.
  • But Firefox doesn't support the mkv format.

So I've searched a lot and found that, I could stream instead of playing the video following these steps.

  • mkv can be converted to an m3u8 using FFmpeg
  • The m3u8 points towards ts segment files
  • then use hls.js on the browser side to play the video

But the catch is I want to do this programmatically.

What I actually want to do:

Steps

  • the client uploads a huge (>1 GB) mkv file to the client's server (server is the client's machine itself)

  • after the upload is done,

  • client requests to play the video.

  • the server starts transcoding and sends the client the m3u8 stream immediately instead of making the client wait for the transcoding to complete.

  • the client should be able to seek the video. (the most IMPORTANT part)

It is possible as Emby and Plex both have implemented it.

I was able to get this to work in chrome as it supports playing some mkv files. I wrote a node js server that accepts Range header and pseudo-streams the video.

A gist

But as I've mentioned Firefox says NO to mkv.

So I tried the hls thing but I couldn't quite get the command to generate the stream and also play on the fly.

I started this on a command line

ffmpeg -i ../stream.mkv -hls_list_size 0 -acodec copy -vcodec copy file.m3u8

and a simple http-server on another shell instance

My index.html file

<html>
    <title>Welcome</title>
    <body>
        <script src="./hls.js"></script>
        <video id="video" width="400" controls></video>
        <script>
            var video = document.getElementById('video');
            if(Hls.isSupported()) {
                var hls = new Hls();
                hls.loadSource('file.m3u8');
                hls.attachMedia(video);
                hls.on(Hls.Events.MANIFEST_PARSED,function() {
                    video.play();
                });
            } else if (video.canPlayType('application/vnd.apple.mpegurl')) {
                video.src = 'file.m3u8';
                video.addEventListener('loadedmetadata',function() {
                    video.play();
                });
            }
        </script>
    </body>
</html>

and while it was running I went ahead and requested the server.

I was able to get the video but It only seeks as far it's converted into the ts files.

And it's random and the video length keeps increasing. It won't play sometimes and after the FFmpeg is done converting to m3u8 the video plays if I refresh the webpage.

I think this has to do with the continuous overwriting of the m3u8 file. is there a way to predetermine the file contents of m3u8 and fill it up?

I want to be able to seek even further and somehow spawn another FFmpeg process to start from that timestamp of the video? How could I approach the seeking part?

So what I'd like to do again is

  • I want to request the server to play a video file
  • It spawns a child process FFmpeg that does the transcoding
  • Sends the client the stream
  • The client should be able to seek to the end and it should play the thing.
like image 292
Phani Rithvij Avatar asked Nov 27 '25 19:11

Phani Rithvij


1 Answers

This problem is actually very difficult. Plex is able to cheat because the file is packaged for each user specifically, and for the most part plex control the player.

The way Plex basically works:

The file is pre analyzed, and a complete manifest is generated. Transcode is kicked off a the start of the file and caches the segments locally. If a seek happens (A segment is requested that is not cached locally) the transcode is canceled, and a new on started at the offset of the segment requested, and the player invalidates any segment it has cached.

If the client cached segments are not invalidated, or a CDN is used, there is no guarantee the the first, post seek, segment will align with its previous segment generated at a later time. This will cause skips and pops at the boundary.

For example seek 1hour out into a movie, than skip back 30 seconds. The two segments are created in the wrong order and will not smoothy play back to back

If you need to support off the shelf players and CDNs, the problem is more difficult because you must be able to generate frame accurate segments on the fly. Frame accurate audio is tricky, especially with codecs the use priming samples. I recommend you look into commercial services like mux.com that that do basically this.

like image 93
szatmary Avatar answered Nov 29 '25 11:11

szatmary



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!