WebRTC with python

Question

I would like to make a streaming server with python/twisted, which receives a WebRTC video stream and then applys some OpenCV algorithms to it.

However I cannot find a python module for WebRTC. How can I send and receive a WebRTC video stream with python/twisted?

Thanks!

Answer 1

I have started putting together the basic blocks needed to create a Python WebRTC endpoint.

One is an asyncio-based Interactive Connectivity Establishment module:

https://github.com/jlaine/aioice

Another one is a Python binding for libsrtp:

https://github.com/jlaine/pylibsrtp

We also need SRTP keying support in the OpenSSL bindings:

https://github.com/pyca/cryptography/pull/4099

On top of this, we can then build an asyncio-based WebRTC implementation:

https://github.com/jlaine/aiortc

I have been able to get both Chrome and Firefox to establish an audio and video stream to a Python-based server.

Answer 2

What you can do is take screen shots continuously and push them to a websocket and allow your twisted server to take a gander at each one as it comes in.

I have modified some common recorders and my version takes Jpeg images and pushes them over a websocket. Feel free to use and modify how you want so that it fits your needs. Source code here. The example I use is pushing down to a libwebsocket server built in C but the same javascript could be used to send to any websocket server.

Answer 3

I've had a similar issue and ended up creating a server that launches a headless chrome instance from which I can access the WebRTC streams, record chunks with a MediaRecorder and finally forward those chunks on via a WebSocket.

I'd love a python based solution so I wouldn't need the intermediary server launching headless chrome instances but haven't been able to find one.

I've been using Node.js and Puppeteer but one could launch the browser instances from your python server and then send the decoded data back via plain old sockets or whatever else tickles your fancy.