How to compress YUYV raw data to JPEG using libjpeg?

Question

I'm looking for an example of how to save a YUYV format frame to a JPEG file using the libjpeg library.

Answer 1

In typical computer APIs, "YUV" actually means YCbCr, and "YUYV" means "YCbCr 4:2:2" stored as Y0, Cb01, Y1, Cr01, Y2 ...

Thus, if you have a "YUV" image, you can save it to libjpeg using the JCS_YCbCr color space.

When you have a 422 image (YUYV) you have to duplicate the Cb/Cr values to the two pixels that need them before writing the scanline to libjpeg. Thus, this write loop will do it for you:

// "base" is an unsigned char const * with the YUYV data
// jrow is a libjpeg row of samples array of 1 row pointer
cinfo.image_width = width & -1; 
cinfo.image_height = height & -1; 
cinfo.input_components = 3; 
cinfo.in_color_space = JCS_YCbCr; 
jpeg_set_defaults(&cinfo); 
jpeg_set_quality(&cinfo, 92, TRUE); 
jpeg_start_compress(&cinfo, TRUE); 
unsigned char *buf = new unsigned char[width * 3]; 
while (cinfo.next_scanline < height) { 
    for (int i = 0; i < cinfo.image_width; i += 2) { 
        buf[i*3] = base[i*2]; 
        buf[i*3+1] = base[i*2+1]; 
        buf[i*3+2] = base[i*2+3]; 
        buf[i*3+3] = base[i*2+2]; 
        buf[i*3+4] = base[i*2+1]; 
        buf[i*3+5] = base[i*2+3]; 
    } 
    jrow[0] = buf; 
    base += width * 2; 
    jpeg_write_scanlines(&cinfo, jrow, 1); 
}
jpeg_finish_compress(&cinfo);
delete[] buf;

Use your favorite auto-ptr to avoid leaking "buf" if your error or write function can throw / longjmp.

Providing YCbCr to libjpeg directly is preferrable to converting to RGB, because it will store it directly in that format, thus saving a lot of conversion work. When the image comes from a webcam or other video source, it's also usually most efficient to get it in YCbCr of some sort (such as YUYV.)

Finally, "U" and "V" mean something slightly different in analog component video, so the naming of YUV in computer APIs that really mean YCbCr is highly confusing.

Answer 2

libjpeg also has a raw data mode, whereby you can directly supply the raw downsampled data (which is almost what you have in the YUYV format). This is more efficient than duplicating the UV values only to have libjpeg downscale them again internally.

To do so, you use jpeg_write_raw_data instead of jpeg_write_scanlines, and by default it will process exactly 16 scanlines at a time. JPEG expects the U and V planes to be 2x downsampled by default. YUYV format already has the horizontal dimension downsampled but not the vertical, so I skip U and V every other scanline.

Initialization:

cinfo.image_width = /* width in pixels */;
cinfo.image_height = /* height in pixels */;
cinfo.input_components = 3;
cinfo.in_color_space = JCS_YCbCr;
jpeg_set_defaults(&cinfo);

cinfo.raw_data_in = true;

JSAMPLE y_plane[16][cinfo.image_width];
JSAMPLE u_plane[8][cinfo.image_width / 2];
JSAMPLE v_plane[8][cinfo.image_width / 2];

JSAMPROW y_rows[16];
JSAMPROW u_rows[8];
JSAMPROW v_rows[8];

for (int i = 0; i < 16; ++i)
{
    y_rows[i] = &y_plane[i][0];
}

for (int i = 0; i < 8; ++i)
{
    u_rows[i] = &u_plane[i][0];
}

for (int i = 0; i < 8; ++i)
{
    v_rows[i] = &v_plane[i][0];
}

JSAMPARRAY rows[] { y_rows, u_rows, v_rows };

Compressing:

jpeg_start_compress(&cinfo, true);

while (cinfo.next_scanline < cinfo.image_height)
{
    for (JDIMENSION i = 0; i < 16; ++i)
    {
        auto offset = (cinfo.next_scanline + i) * cinfo.image_width * 2;
        for (JDIMENSION j = 0; j < cinfo.image_width; j += 2)
        {
            y_plane[i][j] = image.data[offset + j * 2 + 0];
            y_plane[i][j + 1] = image.data[offset + j * 2 + 2];

            if (i % 2 == 0)
            {
                u_plane[i / 2][j / 2] = image_data[offset + j * 2 + 1];
                v_plane[i / 2][j / 2] = image_data[offset + j * 2 + 3];
            }
        }
    }

    jpeg_write_raw_data(&cinfo, rows, 16);
}

jpeg_finish_compress(&cinfo);

I was able to get about a 33% decrease in compression time with this method compared to the one in @JonWatte's answer. This solution isn't for everyone though; some caveats:

• You can only compress images with dimensions that are a multiple of 8. If you have different-sized images, you will have to write code to pad in the edges. If you're getting the images from a camera though, they will most likely be this way.
• The quality is somewhat impaired by the fact that I simply skip color values for alternating scanlines instead of something fancier like averaging them. For my application though, speed was more important than quality.
• The way it's written right now it allocates a ton of memory on the stack. This was acceptable for me because my images were small (640x480) and enough memory was available.

Documentation for libjpeg-turbo: https://raw.githubusercontent.com/libjpeg-turbo/libjpeg-turbo/master/libjpeg.txt