Inserting meta data into a live video stream

Question

I want to achieve the following, but it remains unclear if this is possible.

---

The current scenario:

Someone is streaming a video with audio through OBS to a media server, clients connect through a website.

[OBS Stream/Video Stream] -> [AWS/External Streaming Service] -> Clients

---

The wanted scenario:

capture this stream through a custom media server and manipulate it by injecting certain metadata at certain moments during the livestream. Note the importance of live.

[OBS Stream/Video Stream] -> [My Custom Node.js Server to insert metadata] -> [AWS/External Streaming Service] -> Clients

---

The idea:

The idea is that I want to synchronize the stream to some popup for example. The default protocol stream seems to be RTMP from OBS, but maybe this can be changed. At a given time during the livestream, an html5 videoplayer on the website can read these tags from the livestream (through some additional library such as video.js) and tell the JS application to show some text. In the end, it boils down to synchronizing the video stream to a text stream (eg from a websocket connection)

Potential solutions:

• ID3 tags. I read about ID3 tags in MP3 files, but this does not seem to be what i'm looking as it needs a complete .mp3 file upfront and is not used for streams (Dynamically Inject ID3 in FFMPEG Live Stream). What I want is to dynamically inject metadata into this stream. For example, inject an id at any time (dynamically chosen) which references to a database for example should suffice.

• LTC/Linear Time Code/SMPTE is this possible to embed that in a video stream somehow with node.js? that would enable me to match timings with an id on the client.

Is this possible to do given an incoming video stream with audio? and if so, what is the format of the stream and how do I inject metadata?

---

EDIT: it seems RTMP is not supported without flash in the browser. This is a no-go so I will need to use another stream format such as HLS/FLV?

Answer 1

Sounds like using something like Liquidsoap as your streaming server would do the trick for inserting the metadata into the stream. Plenty of options for manipulating metadata for you to explore.

As for client side decoding you could perhaps use a javascript readable stream within a service worker to split the server output into metadata/video and process as you see fit.

I did a similar thing for processing inband metadata on an infinite mp3 stream which might give you some ideas on where to start. You can find the code for that here