How do i append two video files data to a source buffer using media source api?

Question

I have two videos name v11.webm and v12.webm.

What i want is that these two videos should run seamlessly without any gap.

I am following the media source api approach of appending data to source buffer.

I am referring the demo given on this link

I have modified that example and removed the part of chunking the video and also tried to append data to source buffer file wise.

My code is as follows:

<script>

var video = document.querySelector('video');

window.MediaSource = window.MediaSource || window.WebKitMediaSource;
if (!!!window.MediaSource) {
  alert('MediaSource API is not available');
}

var mediaSource = new MediaSource();

video.src = window.URL.createObjectURL(mediaSource);

mediaSource.addEventListener('webkitsourceopen', function(e) {

    var sourceBuffer = mediaSource.addSourceBuffer('video/webm; codecs="vorbis,vp8"');  

    for(i=1;i<=2;i++)
    {
        (function(i){

          GET('v1'+i+'.webm', function(uInt8Array) {
              var file = new Blob([uInt8Array], {type: 'video/webm'});

              var reader = new FileReader();
              reader.onload = function(e) {
                sourceBuffer.append(new Uint8Array(e.target.result));            
              };
              reader.readAsArrayBuffer(file);

          });
        })(i);
    }

}, false);

mediaSource.addEventListener('webkitsourceended', function(e) {
  logger.log('mediaSource readyState: ' + this.readyState);
}, false);

function GET(url, callback) {
 // alert(url);
  var xhr = new XMLHttpRequest();
  xhr.open('GET', url, true);
  xhr.responseType = 'arraybuffer';
  xhr.send();

  xhr.onload = function(e) {
    if (xhr.status != 200) {
      alert("Unexpected status code " + xhr.status + " for " + url);
      return false;
    }
    callback(new Uint8Array(xhr.response));
  };
}
</script>

Right now the code is not working as desired.

There is an inconsistent mixing of v11.webm and v12.webm file data.

It is not running seamlessly.

Answer 1

Perhaps a bit late, but I was able to figure this out. Your new video is overwriting the old one, because they both begin at time 0. You have to specify that your new video begins at time X before appending it, so your 'webkitsourceopen' event function should be:

/* forget the sourcebuffer variable, we'll just manipulate mediaSource */
mediaSource.addSourceBuffer('video/webm; codecs="vorbis,vp8"');

/* it seems ok to set initial duration 0 */
var duration = 0;
var totalVideos = 2;

/* use this type of loop to ensure that that a single video
   is downloaded and appended before moving on to the next video,
   mediasource seems picky about these being in order */
var i = 0;
(function readChunk_(i){

    /* the GET function already returns a Uint8Array.
       the demo you linked reads it in filereader in order to manipulate it;
       you just want to immediately append it */
    GET('v1' + (i + 1) + '.webm', function(uint8Array){

        if(i == totalVideos) {
            mediaSource.endOfStream();
        } else {

            /* assuming your videos are put together correctly
               (i.e. duration is correct), set the timestamp offset
               to the length of the total video */
            mediaSource.sourceBuffers[0].timestampOffset = duration;

            mediaSource.sourceBuffers[0].append(uint8Array);

            /* set new total length */
            duration = mediaSource.duration;

            readChunk(++i);
        }
    });
})(i);

Now if only MediaSource wasn't so frustratingly picky about the structure of the videos it accepts. I have yet to find a single sample .webm that works besides the same one used in Eric Bidelman's Demo you linked.

EDIT: After doing more testing, the way I set duration may not be correct. If you seem to get exponential duration growth after each append, try setting the timestampoffset to 0 and not changing it. I have no idea why that seems to fix it, and it may be a problem with how I'm generating the webm files.

Answer 2

Just set the sourceBuffer's mode to 'sequence' (default seems to be 'segments')

From the doc: https://developer.mozilla.org/en-US/docs/Web/API/SourceBuffer/mode

sequence: The order in which the segments are appended to the SourceBuffer determines the order in which they are played. Segment timestamps are generated automatically for the segments that observe this order.

In my app, I just set it after adding the the source buffer to the media source:

// Create media source for the stream, and create the source buffer when ready
let self = this;
this._mediaSource = new MediaSource();
this._mediaSource.addEventListener('sourceopen', function () {
  self._sourceBuffer = self.mediaSource.addSourceBuffer(environment.recordingMimeType);
  self._sourceBuffer.mode = 'sequence'; // This is the relevant part
  self._sourceBuffer.addEventListener('error', function (ev) {
    console.error("Source buffer error ??");
    console.error(ev);
  });
});