How to use Media Source Extension (MSE) low-latency mode

Question

I read about MSE has this low-latency mode where provides zero-buffering for decoding. Regardless of the unstable performance this might bring, it should theoretically offers lower latency when used in real-time streaming. Does anybody know what "tricks" to trigger this low-latency mode?

Ref: https://blog.parsecgaming.com/game-streaming-tech-in-the-browser-with-parsec-5b70d0f359bc

Answer 1

This isn't a complete answer, as I'm just learning this myself. It seems that Chromium is using hints from the MP4 stream to determine whether low latency mode should be used.

In video_renderer_impl.cc:

bool ShouldUseLowDelayMode(DemuxerStream* stream) {
  return base::FeatureList::IsEnabled(kLowDelayVideoRenderingOnLiveStream) &&
         stream->liveness() == DemuxerStream::LIVENESS_LIVE;
}

And then in mp4_stream_parser.cc:

// In ISO/IEC 14496-12:2005(E), 8.30.2: ".. If an MP4 file is created in
// real-time, such as used in live streaming, it is not likely that the
// fragment_duration is known in advance and this (mehd) box may be
// omitted."

// We have an unknown duration (neither any mvex fragment_duration nor moov
// duration value indicated a known duration, above.)

// TODO(wolenetz): Investigate gating liveness detection on timeline_offset
// when it's populated. See http://crbug.com/312699
params.liveness = DemuxerStream::LIVENESS_LIVE;

So, if you could generate a stream without durations, it will be assumed to be live and low-delay mode will be used.

There is also some discussion about exposing a mechanism in the future for triggering low latency mode without stream modifications: https://github.com/w3c/media-source/issues/21

Answer 2

https://github.com/whatwg/html/issues/4638 is the current effort being incubated. It is not MSE-specific. Right now, the HTMLMediaElement.latencyHint attribute is in test in Chromium as incubation proceeds. The idea is that it would override any result of the implementation's "live/low-delay" detection heuristic, giving the application more control.