How do I play a stream of H.264 NAL units in a video tag with MSE?

Question

The situation is pretty straight-forward; I am receiving a stream of NAL units via WebSockets. How do I feed them into an HTML5 video tag using MSE?

Research indicates that I should mux the data into a fragmented mp4, but I haven't found any specifics on how to accomplish that. Does anyone have specifics?

Answer 1

If you receive a stream data e.g. hls, nalu h.264...and so on, you can transform and mux that into a fragmented mp4. Setting HTML5 video tag combines with MSE like creating mediaSource, mediaSource.addSourceBuffer, sourceBuffer.appendBuffer. That will play video while fmp4 right feed into buffer.

You may check out https://github.com/ChihChengYang/wfs.js which demonstrates transmuxing NALu h.264 streams from websocket. That works directly on top of a standard HTML5 element and MSE.