How can I fix this YCrCb -> RBG conversion formula?

Question

Im using a formula from this question:

uint8_t *rgbBuffer = malloc(imageWidth * imageHeight * 3);

// .. iterate over height and width

// from ITU-R BT.601, rounded to integers
rgbOutput[0] = (298 * (y - 16) + 409 * cr - 223) >> 8;
rgbOutput[1] = (298 * (y - 16) + 100 * cb + 208 * cr + 136) >> 8;
rgbOutput[2] = (298 * (y - 16) + 516 * cb - 277) >> 8;

Which I assume is based in the ITU-R_BT.601 formula in the wiki article.

However I think the formula is not quite right because the output image looks like this:

How do I fix the formula?

Answer 1

Assuming max values for the first calculation (y == 255 and cr == 255):

rgbOutput[0] = (298 * (255 - 16) + 409 * 255 - 223) >> 8;
rgbOutput[0] = (298 * 239 + 104295 - 223) >> 8;
rgbOutput[0] = (71222 + 104295 - 223) >> 8;
rgbOutput[0] = 175294 >> 8; // 175294 == 0x2ACBE
rgbOutput[0] = 684; // 684 == 0x2AC

The maximum value that rgbOutput[0] can hold is 255. You're attempting to assign 684 to it, resulting in truncation. The actual value assigned to it is 172 (0xAC).

EDIT 1

According to the formula you posted, your first calculation should be as follows:

rgbOutput[0] = ((298 * y) >> 8) + ((409 * cr) >> 8) - 223;

This results in a value of (assuming max values for y and cr) of 480, which results in truncation as well.

EDIT 2

The following equation is said to be recommended:

Using this instead, your first calculation should be like this:

rgbOutput[0] = ((255 * (y - 16)) / 219) + ((179 * (cr - 128)) / 112;

This results in a value of (assuming max values for y and cr) of 480 (the same answer in EDIT 1), which results in truncation as well.

EDIT 3

See answer from @Robert for complete solution.

EDIT 4

When y == 0 and cr == 0, the value that is written to y will also result in truncation unless clamping is performed.