Find the color of connectedComponents in Python OpenCV

Question

Assume I have a numpy array image, which has been binarized: All values are either 0 or 255. In the python variant of opencv, I can do:

conn = cv2.connectedComponentsWithStats(mask, 8, cv2.CV_32S)

And the conn object will tell me:

• How many connected components (labels) there are
• The label of each pixel in the original image
• The bounding box and area of each connected component
• Centroid location for each connected component

Additionally, there is a guarantee that label 0 is the 'background' label, but per the documentation there is no indication of how that background is determined.

Assume my images are black spots on a white background with effectively no guarantees on behavior: Could be one black spot that actually overflows the background (leaving "background corners") or multiple small block dots, or even black dots arrayed in such a way that the create white islands between them, etc.

But I am only interested in my black dots.

Is there a simple, pythonic way to determine the underlying color of each connected segment, when I have no faith in the color of the openCV-provided background component? Alternately: Am I missing something obvious about the connectedComponent feature or the OpenCV API?

(Note: I am aware I can brute force this: Loop through the labels, find the first instance of each label in the provided label array, then probe the corresponding index in the image array. At least I think I can do that-- the documentation is sparse. But this strikes me as extremely inelegant.)

Answer 1

Just as a point of interest, connected components in ImageMagick does list the mean color of the region, since it permits fuzzy-colored region determination (according to a fuzz value), not shown here.

Here is a simple binary example, so the mean color is the actual color:

magick objects.gif -define connected-components:verbose=true -connected-components 4 -auto-level regions.png

Objects (id: bounding-box centroid area mean-color):
  0: 256x171+0+0 119.2,80.8 33117 srgb(0,0,0)
  2: 120x135+104+18 159.5,106.5 8690 srgb(255,255,255)
  3: 50x36+129+44 154.2,63.4 1529 srgb(0,0,0)
  4: 21x23+0+45 8.8,55.9 409 srgb(255,255,255)
  1: 4x10+252+0 253.9,4.1 31 srgb(255,255,255)

The listing is sorted by highest area first. So the "background" would be determined by the largest region area and is listed first.

The result is coded by id number as the gray level, which is stretched to full dynamic range for viewing by -auto-level.

There is also a define argument that can be set to make an output that filters by area and then shows an image containing the resulting regions by their mean color.

magick objects.gif -define connected-components:area-threshold=410 -define connected-components:mean-color=true -connected-components 4 filtered.png

Answer 2

For each connected compenent, you can make a mask and compute the mean with this function: cv2.mean(image, mask)

conn = cv2.connectedComponentsWithStats(input_binary, 8, cv2.CV_32S)
num_labels = conn[0]
labels = conn[1]
stats = conn[2]
centroids = conn[3]

for l in labels:
    mask = np.array(labels, dtype=np.uint8)
    mask[labels==l] = 255 

    mean_label = cv2.mean(input_binary, mask)
    print(mean_label)