GStreamer + V4L2loopback as Chrome compatible webcam

Question

I am trying to create a virtual camera in Chrome using v4l2loopback where the incoming video is H264 via RTP.

I have has some success in getting a GStreamer test video recognized in Chrome with MediaStreamTrack.getSources:

$ sudo modprobe v4l2loopback
$ gst-launch-1.0 videotestsrc ! v4l2sink device=/dev/video0

This works well, Chrome will display the video test source.

However, when I use an incoming h264/RTP source the device does not show up in MediaStreamTrack.getSources. For example:

gst-launch-1.0 -v tcpclientsrc host=<IPADDRESS> port=5000  ! gdpdepay !  rtph264depay ! avdec_h264 ! videoconvert ! v4l2sink device=/dev/video0

What is the reason for this? What would the solution be?

I had thought perhaps this is to do with the video formats and perhaps the correct caps needed to be set through v4l2loopback.

Answer 1

This looks like a bug in gstreamer or v4l2loopback. It is somehow related to how variable frame rate is handled.

I managed to reproduce it in this way:

Start pipeline transmitting video from network to /dev/video0

$ gst-launch-1.0 -v tcpserversrc port=5000 \
    ! gdpdepay ! rtph264depay \
    ! decodebin \
    ! v4l2sink device=/dev/video0

Start pipeline transmitting some video to port 5000

$ gst-launch-1.0 -v videotestsrc \
    ! x264enc ! rtph264pay ! gdppay \
    ! tcpserversink port=5000

Try to get video from /dev/video0

$ gst-launch v4l2src device=/dev/video0 ! autovideosink
...
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Device '/dev/video1' is not a capture device.

Now, note the caps for v4l2sink in the debug log of the first pipeline.

/GstPipeline:pipeline0/GstV4l2Sink:v4l2sink0.GstPad:sink: caps = video/x-raw, format=(string)I420, width=(int)320, height=(int)240, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt601, framerate=(fraction)0/1

It mentions that framerate=(fraction)0/1. In gstreamer's terms this means that frame rate is variable. According to v4l2sink's source code it seems that it feed this same frame rate to v4l2loopback kernel module but v4l2loopback does not understand zero frame rate.
(This is only hypothesis, still need to check if this is what really happens.)

To workaround this bug you can fix frame rate. Just add videorate element to the first pipeline:

$ gst-launch-1.0 -v tcpserversrc port=5000 \
    ! gdpdepay ! rtph264depay \
    ! decodebin \
    ! videorate ! video/x-raw, framerate=25/1 \
    ! v4l2sink device=/dev/video0