Why would cv2.COLOR_RGB2GRAY and cv2.COLOR_BGR2GRAY give different results?

Question

I always thought that converting an image from colored to grayscale is simple: intensity of each pixel would be an average of intensities of each color channel. But I noticed that cv2.COLOR_RGB2GRAY and cv2.COLOR_BGR2GRAY give different results. When I was experimenting with them I also found that it would also be different from the average of intensities of each color channel.

P.S. I was completely baffled when I found that

img_read_as_color[:,:,0]/3+img_read_as_color[:,:,1]/3+img_read_as_color[:,:,2]/3 == (img_read_as_color[:,:,0]+img_read_as_color[:,:,1]+img_read_as_color[:,:,2])/3

but when shown as immage

(img_read_as_color[:,:,0]+img_read_as_color[:,:,1]+img_read_as_color[:,:,2])/3

would look like

img_read_as_color[:,:,0]+img_read_as_color[:,:,1]+img_read_as_color[:,:,2]

Can someone explain to me why this is happening?

My full code:

import matplotlib.pyplot as plt
import cv2

sample = r'G:\Python\knight-mare\screenshots\2020-07-12-02-40-44.jpg'
img_read_as_grayscale = cv2.imread(sample, cv2.IMREAD_GRAYSCALE)
img_read_as_color = cv2.imread(sample, cv2.IMREAD_COLOR)
img_RGB_to_grayscale = cv2.cvtColor(img_read_as_color, cv2.COLOR_RGB2GRAY)
img_BGR_to_grayscale = cv2.cvtColor(img_read_as_color, cv2.COLOR_BGR2GRAY)
plt.imshow(img_read_as_grayscale)
plt.title('img_read_as_grayscale')
plt.show()
plt.imshow(img_read_as_color)
plt.title('img_read_as_color')
plt.show()
plt.imshow(img_RGB_to_grayscale)
plt.title('img_RGB_to_grayscale')
plt.show()
plt.imshow(img_BGR_to_grayscale)
plt.title('img_BGR_to_grayscale')
plt.show()

channel_avg_div_separately = img_read_as_color[:,:,0]/3+img_read_as_color[:,:,1]/3+img_read_as_color[:,:,2]/3
channel_avg_div_together = (img_read_as_color[:,:,0]+img_read_as_color[:,:,1]+img_read_as_color[:,:,2])/3
channel_sum = img_read_as_color[:,:,0]+img_read_as_color[:,:,1]+img_read_as_color[:,:,2]
plt.imshow(channel_avg_div_separately)
plt.title('channel_avg_div_separately')
plt.show()
plt.imshow(channel_avg_div_together)
plt.title('channel_avg_div_together')
plt.show()
plt.imshow(channel_sum)
plt.title('channel_sum')
plt.show()

Answer 1

1. RGB->gray conversion actually isn't an average -- different channels are weighed differently. Specifically:

gray_pixel = 0.114 * blue_pixel + 0.299 * red_pixel + 0.587 * green_pixel

This is also mentioned in the documentation. Thus, it's expected that RGB2GRAY and BGR2GRAY give different results.

2. Regarding the discrepancy between sum-then-divide and divide-then-sum approaches, i.e. between

img_read_as_color[:,:,0]/3+img_read_as_color[:,:,1]/3+img_read_as_color[:,:,2]/3

and

(img_read_as_color[:,:,0]+img_read_as_color[:,:,1]+img_read_as_color[:,:,2])/3

Recall that cv2.imread returns a uint8 numpy array. Thus, the latter operation (where all channels are combined together prior to division) results in an overflow (in fact, ipython3 gives me a runtime warning in this case). Overflow-like artifacts are also visible in images labeled channel_avg_div_together and channel_sum.

Answer 2

Well, firstly the conversion is not a simple average let alone a linear transform. The formula for calculating is RGB[A] to Gray:Y←0.299⋅R+0.587⋅G+0.114⋅B (OpenCV Docs)

Apparently OpenCV uses the same formula for conversion to greyscale whether its BGR or RGB used as input, but the channels order is retained when using the formula so the wrong order passed for conversion will lead to wrong results.

A good example of this is in a forum post i came across recently where the author compares the results after the conversion from RGB and BGR. The lower right corner pixel has the following values.(Image credits to the linked post/author)

CV_BGR2GRAY: Lower right corner: 0.1140 * 163 + 0.5870 * 182 + 0.2989 * 203 ≈ 186

CV_RGB2GRAY: Lower right corner: 0.1140 * 203 + 0.5870 * 182 + 0.2989 * 163 ≈ 179

So tl;dr: if you pass the wrong order of channels, the converted output will differ.

Answer 3

I think the problem is with the MatPlotLib .imshow() method. In my experiments both:

img_read_as_grayscale = cv2.imread(sample, cv2.IMREAD_GRAYSCALE)

and

img_BGR_to_grayscale = cv2.cvtColor(img_read_as_color, cv2.COLOR_BGR2GRAY)

will be properly displayed as a grayscale image when using cv2.imshow() but will be displayed with a weird blue and green color with plt.imshow().

You can fix this by specifying cmap='gray'

plt.imshow(img_BGR_to_grayscale, cmap='gray')