Stream Video using Twilio from IP Camera RTSP

Question

All of Twilio's examples for their Programmable Video service that I've been able to find either demonstrate screen sharing or webcam media streams. Can someone point me to an example that streams video from an RTSP stream provided by an IP Camera?

I've been able to find examples of and experiment with this behavior using Kurento, so I figured Twilio-Video might expose the same. See https://github.com/lulop-k/kurento-rtsp2webrtc

Answer 1

Did you take a look at this?

It's an interesting, well-written article on the matter.

From the link in question:

WebRTC Media Gateways for media interoperability For integrating an IP camera with a WebRTC application you first need to achieve media interoperability. This means that the media stream provided by the camera needs to be made compatible with the WebRTC codecs and formats supported by browsers. This means to translate whatever the IP camera speaks into whatever the WebRTC browser supports. For this to happen, typically a piece of technology called a WebRTC Media Gateway is required. For understanding what such a gateway does, observe the following.

Most IP cameras available in the market (excluding exotic ones) publish media through any of these mechanisms:

RTSP/H.264: These types of cameras are typical for video surveillance applications. They use the RTSP protocol for establishing an RTP media session. In other words, signaling takes place through RTSP while media transport itself is based on plain RTP. Different camera vendors may support different RTP profiles but, for most of the cameras I've seen, the AVP is the only available option. In these cameras, and also typically, H.264 is the only option for the codec.

HTTP/MJPEG: These cameras use HTTP streaming for signaling and transport and encode video as a sequence of JPEG pictures. The hardware for these cameras is simpler and requires fewer resources to operate. This is why they are often used when battery consumption or weight is an issue (e.g. robotics, drones, etc.) As a drawback, the video quality tends to decrease significantly.

Doing it right with Kurento Media Server

The Kurento Media Server toolbox makes possible to create rich WebRTC Media Gateways in a flexible way and programming in Java or JavaScript if you want. For an introduction on Kurento Media Server technologies, just take a look to the documentation. Implementing a WebRTC Media Gateway for interoperating with IP cameras in Kurento is trivial and safe. You need only take into consideration three aspects:

Kurento Media Server PlayerEndpoint supports reading video streams from different types of sources including RTSP/RTP and HTTP/MJPEG. In other words, the PlayerEndpoint is capable of managing the capture of media from the IP camera.

Kurento Media Server WebRtcEndpoint supports publishing media streams to WebRTC browsers with full termination of RTCP feedback. This means that, every time a PLI packet is received, the WebRtcEndpoint shall command the VP8 encoder to generate a new key frame. This also means that REMB feedback and congestion control shall be honored by commanding the VP8 encoder to decrease its quality.

Kurento Media Server agnostic media capability performs, transparently for the developer, all the appropriate trans-codifications when two incompatible media elements are connected. Hence, in this case, just by connecting the PlayerEndpoint source to the WebRtcEndpoint sink the H.264/MJPEG to VP8 transcoding shall take place.

The souce code of a JavaScript application implementing this logic is sketched below:

    var pipeline = ...//Use Kurento Client API for obtaining your pipeline.

//Create the PlayerEndpoint for receiving from the IP camera. HTTP and RTSP uris are supportd
pipeline.create("PlayerEndpoint", {uri: "rtsp://your.rtsp.address"}, function(error, playerEndpoint){

    //Create the WebRtcEndpoint
    pipeline.create("WebRtcEndpoint", function(error, webRtcEndpoint){
    
    //If working with trickle ice, some code for candidate management is required here.
    
        //Connect playerEndpoint to webRtcEndpoint. This connection activates the agnostic media
        //capability and the appropriate transcodings are configured and activated.
    playerEndpoint.connect(webRtcEndpoint, function(error){
        
                //Media starts flowing ... enjoy
        player.play(function(error){
        });
    });
    });
});

If you want a fully working example in JavaScript, you can take a look to this GitHub repository

This should provide a solution to your needs, comment if you have any problems.

Good luck!