MediaSource API - append/concatenate multiple videos together into a single buffer

Question

UPDATE:

SO I was able to get this to work by using the offsetTimestamp property (incrementing it after appending each video).

My questions now: 1) Why isn't this done properly when setting the MediaSource.mode to sequence?

2) Why is my MediaSource.duration always "Infinity" and not the correct duration?

---

I'm trying to use the MediaSource API to append multiple video files and play them seamlessly as if it were 1 video.

I've properly transcoded my videos according to the spec (DASH-MPEG) and when playing them individually, they work fine.

However, when I try to append multiple, I run into issues where the segments overwrite one another, incorrect duration, etc. Even though everything seems to be executing as expected.

I've tried playing around with the offsetTimestamp, but according to the documentation setting MediaSource.mode to 'sequence' should automatically handle this. Also, for some reason, MediaSource.duration always seems to be 'Infinity' even after appending a segment.

Here is my code:

<script>
 function downloadData(url, cb) {
    console.log("Downloading " + url);

    var xhr = new XMLHttpRequest;
    xhr.open('get', url);
    xhr.responseType = 'arraybuffer';
    xhr.onload = function () {
        cb(new Uint8Array(xhr.response));
    };
    xhr.send();
}

if (MediaSource.isTypeSupported('video/mp4; codecs="avc1.64001E"')) {
    console.log("mp4 codec supported");
}

var videoSources = [
    "{% static 'mp4/ff_97.mp4' %}",
    "{% static 'mp4/ff_98.mp4' %}",
    "{% static 'mp4/ff_99.mp4' %}",
    "{% static 'mp4/ff_118.mp4' %}"
]

var mediaSource = new MediaSource();


mediaSource.addEventListener('sourceopen', function(e) {


    var sourceBuffer = mediaSource.addSourceBuffer('video/mp4; codecs="avc1.64001E"');
    sourceBuffer.mode = 'sequence';

    console.log('SourceBuffer mode set to ' + sourceBuffer.mode);

    sourceBuffer.addEventListener('updateend', function(e) {
        console.log('Finished updating buffer');
        console.log('New duration is ' + String(mediaSource.duration));

        if (videoSources.length == 0) {
            mediaSource.endOfStream();
            video.currentTime = 0;
            video.play();
            return;
        }

        downloadData(videoSources.pop(), function(arrayBuffer) {
            console.log('Finished downloading buffer of size ' + String(arrayBuffer.length));
            console.log('Updating buffer');
            sourceBuffer.appendBuffer(arrayBuffer);
        });

        console.log('New duration: ' + String(mediaSource.duration));

    });

    downloadData(videoSources.pop(), function(arrayBuffer) {
        console.log('Finished downloading buffer of size ' + String(arrayBuffer.length));
        console.log('Updating buffer');
        sourceBuffer.appendBuffer(arrayBuffer);
    });



}, false);

var video = document.querySelector('video');
video.src = window.URL.createObjectURL(mediaSource);

And here is the logs:

mp4 codec supported
(index):78 SourceBuffer mode set to sequence
(index):45 Downloading /static/mp4/ff_118.mp4
(index):103 Finished downloading buffer of size 89107
(index):104 Updating buffer
(index):81 Finished updating buffer
(index):82 New duration is Infinity
(index):45 Downloading /static/mp4/ff_99.mp4
(index):98 New duration: Infinity
(index):92 Finished downloading buffer of size 46651
(index):93 Updating buffer
(index):81 Finished updating buffer
(index):82 New duration is Infinity
(index):45 Downloading /static/mp4/ff_98.mp4
(index):98 New duration: Infinity
(index):92 Finished downloading buffer of size 79242
(index):93 Updating buffer
(index):81 Finished updating buffer
(index):82 New duration is Infinity
(index):45 Downloading /static/mp4/ff_97.mp4
(index):98 New duration: Infinity
(index):92 Finished downloading buffer of size 380070
(index):93 Updating buffer
(index):81 Finished updating buffer
(index):82 New duration is Infinity

Answer 1

2) Why is my MediaSource.duration always "Infinity" and not the correct duration?

You need to call MediaSource.endOfStream() in order for the MediaSource object to calculate the actual duration of segments in its SourceBuffer. I see that you are doing this, but it looks like you're trying to access MediaSource.duration before calling endOfStream(). I suggest you read up on the end of stream algorithm in the MSE Spec, you'll notice that it will lead to invoking the duration change algorithm.

If you want to have your <video> element report a duration before calling MediaSource.endOfStream(), you can actually set a value using MediaSource.duration based on your own estimate of segments appended.

1) Why isn't this done properly when setting the MediaSource.mode to sequence?

As far as I know, it should do. But I have preferred the explicit timestampOffset approach myself as it provides more flexibility when wanting to append segments far ahead in the buffer (ie. if the user seeks way ahead of the current buffer end, you'll want to start loading+appending after a gap). Although I appreciate that seeking my not be a requirement in your use-case.