Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Which YCbCr matrix to use? BT.709 or BT.601

I'm working on a video player project on iOS.

It uses AVFoundation to extract CVPixelBuffer from a video file and then send that buffer to OpenGL as textures.

The proof-of-concept code is inspired by Apple's sample code. The AVFoundation provides each frame in YCbCr color space and it need to be transformed into RGB to render in OpenGL. This transform seems to have multiple transform matrix options depending on different YCbCr standard(e.g. ITU-R BT.709, ITU-R BT.601). The sample code determines which one to use by following code:

CFTypeRef colorAttachments = CVBufferGetAttachment(pixelBuffer, kCVImageBufferYCbCrMatrixKey, NULL);
if (colorAttachments == kCVImageBufferYCbCrMatrix_ITU_R_601_4) {
    _preferredConversion = kColorConversion601;
}
else {
    _preferredConversion = kColorConversion709;
}

However, I'm using swift and the return colorAttachment is of type Unmanaged<CFTypeRef> while the constant kCVImageBufferYCbCrMatrix_ITU_R_601_4 is of type CFString so they can't be equaled directly. I did some research and ended up with:

CFEqual(colorAttachments, kCVImageBufferYCbCrMatrix_ITU_R_601_4) // returns false
CFEqual(colorAttachments, kCVImageBufferYCbCrMatrix_ITU_R_709_2) // returns false too!!
//-----------------------------------------
CFGetType(colorAttachments) // returns 1
CFStringGetType() // returns 7, note kCVImageBufferYCbCrMatrix_ITU_R_601_4 is of type CFString
// so I still can't check their equality 
// because the retrieved  colorAttachments is not of type CFString at all

I tried two transforms one-by-one by hardcoding the matrix and the result (rendered scene) seems no difference to human eye which is predictable because the two transform matrixes are not differed to much.

My questions:

  1. How to determine which transform to use?
  2. If it's impossible to solve [1.], can I hard code either one? what's the consequence of doing so?
like image 706
Nandin Borjigin Avatar asked Dec 19 '16 07:12

Nandin Borjigin


1 Answers

Using takeUnretainedValue() will give you a CFTypeRef. This then needs to be downcast to a CFString. For example, your code could look like this:

if let colorAttachment = CVBufferGetAttachment(image, kCVImageBufferYCbCrMatrixKey, nil)?.takeUnretainedValue(),
    CFGetTypeID(colorAttachment) == CFStringGetTypeID() {
    let colorAttachmentString = colorAttachment as! CFString
    print(colorAttachmentString)
    print(colorAttachmentString == kCVImageBufferYCbCrMatrix_ITU_R_601_4)
    print(colorAttachmentString == kCVImageBufferYCbCrMatrix_ITU_R_709_2)
}

Which prints:

ITU_R_601_4
true
false
like image 165
Jan Rüegg Avatar answered Oct 21 '22 17:10

Jan Rüegg