Capturing USB type C video through software

Question

So, I was just thinking about something: Is it theoretically possible to be able to connect something such as a Switch with USB-C to a PC and do something similar to emulating a capture card through software. I did a bit of reading on the specifications for type C, and I wasn't sure if something like this was possible. I assume something like this would have to be fairly low-level to do as well.

Answer 1

USB protocol is organized in device classes, i.e. Mass Storage Device Class, Communication Device Class ( CDC ) and also USB Video Device Class ( UVC ) almost any modern camera device implements this UVC class and you can capture the stream of UVC devices with software like vlc

https://en.wikipedia.org/wiki/USB_video_device_class

https://ubuntuforums.org/showthread.php?t=143732 - HOW TO: Use VLC to capture video from your webcam to a file

USB-C standard is the physical layout of the connector, it does not change the fundamental organizational structure of USB standard like its differentiation in device classes and so there are UVC devices with USB-C connector, search internet for i.e. 'webcam USB-C' however USB-C extends the functionality of the port by the Alternate Modes and so also uses different port controllers with different firmware and kernel modules

Alt Mode is a functional extension of USB-C which enables the USB connection to carry non-USB signals.

source : https://blog.startech.com/post/tech-talk-using-usb-c-and-displayport-over-alt-mode/

https://en.wikipedia.org/wiki/USB-C

what i do not completely understand is if you want to capture HDMI streams over USB-C via USB-C HDMI Alternate Mode ( https://en.wikipedia.org/wiki/HDMI#HDMI_Alternate_Mode_for_USB_Type-C ). In this case there is no USB Video Device protocol ( HDMI is a different protocol ) and the situation is more complicated, see https://superuser.com/questions/1218848/hdmi-input-through-usb-c and https://superuser.com/questions/1192638/whats-up-with-hdmi-and-displayport-over-usb-c

there are active HDMI-to-USB video capture cards that convert the HDMI stream into a video format that is then transmitted over USB Video Class protocol or directly into raw pixel data . This raw pixel data is also wrapped in USB Video Class protocol ( https://devtalk.nvidia.com/default/topic/970330/how-to-handle-raw-from-usb-cameras/?offset=2 ) :

Other video capture solutions compress your video into a container like H.264 or Motion JPG so it’s small enough to fit across a USB 2.0 bus (a medium with a maximum data transfer rate of 60 MB/s). When it arrives on your computer, the CPU and/or GPU need to work to decompress this image, then re-encode it with your encoding settings. And with this type of pre-compressed video, it’s never possible to get back the full detail that was lost in compression.

By contrast, our AV.io family of HDMI capture cards send raw pixel data across to your computer over USB 3.0 (max transfer of 640 MB/s). When your software receives this stream, it doesn’t need to decode it, so there’s less heavy lifting.

source : https://www.epiphan.com/blog/easiest-hdmi-to-usb-converter/

that raw pixel data can be transfered over USB-C in HDMI Alternate Mode should make it possible to capture a HDMI stream directly from USB-C, however there is a major drawback :

problematic is that in USB(-C) protocol a machine can either be a host or device or dual-role :

Devices may be hosts (DFP: Downstream-facing port) or peripherals (UFP: Upstream-facing port). Some, such as mobile phones, can take either role depending on what kind is detected on the other end. These types of ports are called Dual-Role-Data (DRD) ports, which was known as USB On-The-Go in previous specification.[12]

in a USB tree can only be one host and in most cases a computer is in this role as host. USB-C input or peripheral devices like displays are in device or peripheral mode. If you are not able to set your computer into device or peripheral mode you will not be able to capture HDMI signals over USB-C because of the USB protocol

critical is the firmware and kernel module / driver of the USB-C port controller

the linux USB-C kernel module is in https://github.com/torvalds/linux/blob/master/drivers/usb/host/xhci.c and https://cateee.net/lkddb/web-lkddb/TYPEC_TCPM.html and it is defined as USB-C host or downstream-facing port only, so it is impossible to use it as HDMI sink or upstream-facing port and so it is impossible to use it as input and capture from it ...

the inability to set a USB-C port into device mode or upstream-facing port mode is the reason why external adapters have to be used

an example for a USB-C device that is a USB peripheral or upstream-facing port mode is i.e. https://9to5mac.com/2016/06/04/lg-27ud88-w-usb-c-4k-monitor-macbook-review/ so in this devices runs firmware that processes HDMI streams over an USB-C connector, however in USB protocol it implements an upstream-facing port, what a computer or laptop can not implement due to its hardware restrictions that implements a USB host or downstream-facing port ..