How to convert an FFMPEG AVFrame in YUVJ420P to AVFoundation cVPixelBufferRef?

Question

I have an FFMPEG AVFrame in YUVJ420P and I want to convert it to a CVPixelBufferRef with CVPixelBufferCreateWithBytes. The reason I want to do this is to use AVFoundation to show/encode the frames.

I selected kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange and tried converting it since the AVFrame has the data in three planes Y480 Cb240 Cr240. And according to what I've researched this matches the selected kCVPixelFormatType. By being biplanar I need to convert it into a buffer that contains Y480 and CbCr480 Interleaved.

I tried to create a buffer with 2 planes:

• frame->data[0] on the first plane,
• frame->data[1] and frame->data[2] interleaved on the second plane.

However, I'm getting return error -6661 (invalid a) from CVPixelBufferCreateWithBytes:

"Invalid function parameter. For example, out of range or the wrong type."

I don't have expertise on image processing at all, so any pointers to documentation that can get me started in the right approach to this problem are appreciated. My C skills aren't top of the line either so maybe I'm making a basic mistake here.

    uint8_t **buffer = malloc(2*sizeof(int *));
    buffer[0] = frame->data[0];
    buffer[1] = malloc(frame->linesize[0]*sizeof(int));
    for(int i = 0; i<frame->linesize[0]; i++){
        if(i%2){
            buffer[1][i]=frame->data[1][i/2];
        }else{
            buffer[1][i]=frame->data[2][i/2];
        }
    }

    int ret = CVPixelBufferCreateWithBytes(NULL, frame->width, frame->height, kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange, buffer, frame->linesize[0], NULL, 0, NULL, cvPixelBufferSample)

The frame is the AVFrame with the rawData from FFMPEG Decoding.

Answer 1

My C skills aren't top of the line either so maybe im making a basic mistake here.

You're making several:

• You should be using CVPixelBufferCreateWithPlanarBytes(). I do not know if CVPixelBufferCreateWithBytes() can be used to create a planar video frame; if so, it will require a pointer to a "plane descriptor block" (I can't seem to find the struct in the docs).
• frame->linesize[0] is the bytes per row, not the size of the whole image. The docs are unclear, but the usage is fairly unambiguous.
• frame->linesize[0] refers to the Y plane; you care about the UV planes.
• Where is sizeof(int) from?
• You're passing in cvPixelBufferSample; you might mean &cvPixelBufferSample.
• You're not passing in a release callback. The documentation does not say that you can pass NULL.

Try something like this:

size_t srcPlaneSize = frame->linesize[1]*frame->height;
size_t dstPlaneSize = srcPlaneSize *2;
uint8_t *dstPlane = malloc(dstPlaneSize);
void *planeBaseAddress[2] = { frame->data[0], dstPlane };

// This loop is very naive and assumes that the line sizes are the same.
// It also copies padding bytes.
assert(frame->linesize[1] == frame->linesize[2]);
for(size_t i = 0; i<srcPlaneSize; i++){
    // These might be the wrong way round.
    dstPlane[2*i  ]=frame->data[2][i];
    dstPlane[2*i+1]=frame->data[1][i];
}

// This assumes the width and height are even (it's 420 after all).
assert(!frame->width%2 && !frame->height%2);
size_t planeWidth[2] = {frame->width, frame->width/2};
size_t planeHeight[2] = {frame->height, frame->height/2};
// I'm not sure where you'd get this.
size_t planeBytesPerRow[2] = {frame->linesize[0], frame->linesize[1]*2};
int ret = CVPixelBufferCreateWithPlanarBytes(
        NULL,
        frame->width,
        frame->height,
        kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange,
        NULL,
        0,
        2,
        planeBaseAddress,
        planeWidth,
        planeHeight,
        planeBytesPerRow,
        YOUR_RELEASE_CALLBACK,
        YOUR_RELEASE_CALLBACK_CONTEXT,
        NULL,
        &cvPixelBufferSample);

Memory management is left as an exercise to the reader, but for test code you might get away with passing in NULL instead of a release callback.