MediaSource Buffer works on only one client

Question

So I am trying to a build a livestreaming application, and ran into a strange issue.

So I am using getUserMedia to capture video from a user, the user is then marked as the broadcaster. I then use MediaRecorder to get the actual video data from the MediaStream and send it over through a websocket.

The websocket simply broadcasts the video data to all of the connected clients, for some reason however, it only plays correctly on the broadcaster's player, but when I try to play the same exact stream from a different client, it just gives me this error:

Uncaught DOMException: Failed to execute 'appendBuffer' on 'SourceBuffer': This SourceBuffer has been removed from the parent media source.

When I look in chrome://media-internals I see this error:

00:00:00 94 error Unexpected element ID 0x8c 00:00:00 94 error Append: stream parsing failed. Data size=33926 append_window_start=0 append_window_end=inf 00:00:00 94 pipeline_error CHUNK_DEMUXER_ERROR_APPEND_FAILED

The scripts and everything is of course the same on both clients. The only thing that differs and that I thought could be the culprit was the fact that the broadcaster's stream did not begin immediately, since the broadcaster first needs to send video data to the websocket server for the server to relay anything back. Non-broadcaster clients however kind of jump in "the middle" of the stream.

The only other difference is that the broadcast is downloading and uploading video data via the same socket, but I don't see how that would play into the issue.

EDIT: After running a test where both clients were connected to the websocket WITHOUT video being streamed, and then running the stream after, both clients worked, meaning the issue does stem from the other clients jumping into the stream mid-way, how would this be fixed?

I'm new to all this however so I'm not sure if it's good reasoning. For reference, this is the script:

var socket = new WebSocket('websocket');
socket.binaryType = 'arraybuffer';

var broadcastMs = new MediaSource();

var video = document.querySelector("#broadcast");
video.src = window.URL.createObjectURL(broadcastMs);

var msReady = false;
var sourceBuffer = false;
var queue = [];

broadcastMs.addEventListener('sourceopen', function(e)
{
    sourceBuffer = broadcastMs.addSourceBuffer('video/webm; codecs="opus,vp8"');

    sourceBuffer.addEventListener('update', function()
    {
        if ( queue.length > 0 && !sourceBuffer.updating )
            sourceBuffer.appendBuffer(queue.shift());
    });

    msReady = true;     
});

socket.onmessage = function(ev)
{
    setTimeout(function()
    {
        if ( msReady )
        {
            if ( sourceBuffer.updating )
                queue.push(ev.data);
            else
                sourceBuffer.appendBuffer(ev.data);
        }
    }, 50);
};

Answer 1

You cannot just needle-drop into a WebM stream. You must first send several parts to initialize the stream.

Snag a copy of EBML Viewer. https://code.google.com/archive/p/ebml-viewer/downloads (EBML is a standard binary format that Matroska is based on. WebM is a subset of Matroska.) Open up a WebM file. You'll see a Segment element which will contain a bunch of data that configures the stream. (Tracks, codecs, etc.) All of that data, up to the start of the first cluster, must be sent into the MediaSource first. After that, you can start on any segment that contains a keyframe.

I should point out that what you're doing is a solved problem. Look into WebRTC for a much more efficient way to do this. You get less control... but a lot of things for free.