Mapping pixel formats from CVPixelBuffer to their equivalent V4L

Question

I need to map a range of OSX CoreVideo pixel formats as enumerated in CVPixelBuffer.h to their equivalents in V4L. For example, kCVPixelFormatType_24RGB would map to V4L2_PIX_FMT_RGB24.

I have tried to match using fourcc, but the definitions for OSX and V4L do not match. Besides inspecting their exact layout and matching them manually is there a way programatically or an information table that will show me for example where kCVPixelFormatType_422YpCbCr8 would map to?

Answer 1

Core Video was definitely not designed to be interoperable with non-Apple systems, and Core Video compatibility has never been a goal for V4L, so there are some pixel formats that only exist in one library. Some formats, like RGB24, exist in both because they are common formats that both Linux and OS X/iOS need to deal with. To confirm that two pixel formats are the same, you'll need to expect their layout and match them manually. There's no function in either library to do the mapping for you.

That said, there are pretty good comments in CVPixelBuffer.h and videodev2.h that allow you to match pixel formats with confidence. The key to mapping some of the formats is to understand that YUV and Y'CbCr are often used interchangeably. As per Wikipedia,

The scope of the terms Y'UV, YUV, YCbCr, YPbPr, etc., is sometimes ambiguous and overlapping. Historically, the terms YUV and Y'UV were used for a specific analog encoding of color information in television systems, while YCbCr was used for digital encoding of color information suited for video and still-image compression and transmission such as MPEG and JPEG. Today, the term YUV is commonly used in the computer industry to describe file-formats that are encoded using YCbCr.

V4L also has excellent documentation on its pixel formats, which can help to match the comments in CVPixelBuffer.h to the V4L formats.

You specifically asked what kCVPixelFormatType_422YpCbCr8 would map to. Given this information, it should map to V4L2_PIX_FMT_UYVY. Below is a map of Core Video pixel Formats to V4L formats constructed using this information. If a format does not show up in the table, that means it's not supported in both libraries. This information is not based on any actual testing, so it could be incorrect.

Note that only 8-bit formats appear to be common between the two libraries. Even in cases where they appear to support the same 16 or 32-bit formats, they do not due to endian differences. For example, kCVPixelFormatType_16Gray does not map to V4L2_PIX_FMT_Y16, because the former is big-endian, and the latter little-endian.

| Core Video | Video4Linux | | kCVPixelFormatType_16BE555 | V4L2_PIX_FMT_RGB555X | | kCVPixelFormatType_16LE555 | V4L2_PIX_FMT_RGB555 | | kCVPixelFormatType_16BE565 | V4L2_PIX_FMT_RGB565X | | kCVPixelFormatType_16LE565 | V4L2_PIX_FMT_RGB565 | | kCVPixelFormatType_24RGB | V4L2_PIX_FMT_RGB24 | | kCVPixelFormatType_24BGR | V4L2_PIX_FMT_BGR24 | | kCVPixelFormatType_32ARGB | V4L2_PIX_FMT_RGB32 | | kCVPixelFormatType_32BGRA | V4L2_PIX_FMT_BGR32 | | kCVPixelFormatType_422YpCbCr8 | V4L2_PIX_FMT_UYVY | | kCVPixelFormatType_420YpCbCr8Planar* | V4L2_PIX_FMT_YUV420 | | kCVPixelFormatType_422YpCbCr8_yuvs | V4L2_PIX_FMT_YUYV |

* The Core Video version contains a big-endian header consisting of where the planes are located and how many bytes per row each plane has. The V4L version does not have this, so you'll have to remove it to go from CoreVideo to V4L, and add it to go from V4L to Core Video.