Receive webRTC video stream using python OpenCV in real-time

Question

Hi,
I am creating a pipeline where I need to access data from the camera and do some OpenCV algorithms in it. I am able to send the video from the source using webRTC. https://lostechies.com/derickbailey/2014/03/13/build-a-local-webcam-with-webrtc-in-less-than-20-lines/

But, What I need help with is how to receive the video stream in Python and do the processing. How can I access the video feed from a webRTC stream to the Python backend?

This is the javascript code running.

(function(){
  var mediaOptions = { audio: false, video: true };

  if (!navigator.getUserMedia) {
      navigator.getUserMedia = navigator.getUserMedia || navigator.webkitGetUserMedia || navigator.mozGetUserMedia || navigator.msGetUserMedia;
  }

  if (!navigator.getUserMedia){
    return alert('getUserMedia not supported in this browser.');
  }

  navigator.getUserMedia(mediaOptions, success, function(e) {
    console.log(e);
  });

  function success(stream){
    var video = document.querySelector("#player");
    video.src = window.URL.createObjectURL(stream);
  }
})();
 

I need help in receiving the video from this Javascript using Python.

Answer 1

I'm the author of aiortc. Have you checked out the server example, as it illustrates how to process video using OpenCV?

https://github.com/jlaine/aiortc/tree/master/examples/server

Answer 2

https://webrtchacks.com/webrtc-cv-tensorflow/ shows a fairly in-depth tutorial for doing WebRTC + tensorflow. You can probably swap out tensorflow for opencv easily. This captures a frame from the webcam and sends it using HTTP every once in a while. If you want to go more realtime than that you will have to use WebRTC on the server, e.g. using https://github.com/jlaine/aiortc