Detect multiple colored regions in image and produce individual crops for each

Question

I have an image like this:

And I want to crop the image anywhere there is red.

So with this image I would be looking to produce 4 crops:

Obviously I first need to detect anywhere there is red in the image. I can do the following:

import cv2
import numpy as np
from google.colab.patches import cv2_imshow

## (1) Read and convert to HSV
img = cv2.imread("my_image_with_red.png")
hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)

## (2) Find the target red region in HSV
hsv_lower = np.array([0,50,50])
hsv_upper = np.array([10,255,255])
mask = cv2.inRange(hsv, hsv_lower, hsv_upper)

## (3) morph-op to remove horizone lines
kernel = np.ones((5,1), np.uint8)
mask2 = cv2.morphologyEx(mask, cv2.MORPH_OPEN,  kernel)


## (4) crop the region
ys, xs = np.nonzero(mask2)
ymin, ymax = ys.min(), ys.max()
xmin, xmax = xs.min(), xs.max()

croped = img[ymin:ymax, xmin:xmax]

pts = np.int32([[xmin, ymin],[xmin,ymax],[xmax,ymax],[xmax,ymin]])
cv2.drawContours(img, [pts], -1, (0,255,0), 1, cv2.LINE_AA)


cv2_imshow(croped)
cv2_imshow(img)
cv2.waitKey()

Which gives the following result:

The bounding box covers the entire area containing red.

How can I get bounding boxes around each red piece of the image? I have looked into multiple masks but this doesn't seem to work.

What I am looking for is:

• detect each red spot in the image;
• return boundaries on each red dot;
• use those boundaries to produce 4 individual crops as new images.

Answer 1

There are currently several problems:

1. If you look at your mask image, you will see that all traces of red are captured on the mask including the small noise. You're currently using np.nonzero() which captures all white pixels. This is what causes the bounding box to cover the entire area. To fix this, we can tighten up the lower hsv threshold to get this resulting mask:

Note there are still a lot of small blobs. Your question should be rephrased to

How can I crop the large red regions?

If you want to capture all red regions, you will obtain much more then 4 crops. So to remedy this, we will perform morphological operations to remove the small noise and keep only the large pronounced red regions. This results in a mask image that contains the large regions

2. You do not require multiple masks

How can I get bounding boxes around each red piece of the image?

You can do this using cv2.findContours() on the mask image to return the bounding rectangles of each red dot.

Oh? This is not your desired result. Since your desired result has some space surrounding each red dot, we also need to include a offset to the bounding rectangle. After adding an offset, here's our result

Since we have the bounding rectangles, we can simply use Numpy slicing to extract and save each ROI. Here's the saved ROIs

---

So to recap, to detect each red spot in the image, we can use HSV color thresholding. Note this will return all pixels which match this threshold which may be different from what you expect so it is necessary to perform morphological operations to filter the resulting mask. To obtain the bounding rectangles on each red blob, we can use cv2.findContours() which will give us the ROIs using cv2.boundingRect(). Once we have the ROI, we add a offset and extract the ROI using Numpy slicing.

import cv2
import numpy as np

image = cv2.imread("1.png")
original = image.copy()
hsv = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)

hsv_lower = np.array([0,150,50])
hsv_upper = np.array([10,255,255])
mask = cv2.inRange(hsv, hsv_lower, hsv_upper)

kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (3,3))
opening = cv2.morphologyEx(mask, cv2.MORPH_OPEN, kernel, iterations=1)
close = cv2.morphologyEx(opening, cv2.MORPH_CLOSE, kernel, iterations=1)

cnts = cv2.findContours(close, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
offset = 20
ROI_number = 0
for c in cnts:
    x,y,w,h = cv2.boundingRect(c)
    cv2.rectangle(image, (x - offset, y - offset), (x + w + offset, y + h + offset), (36,255,12), 2)
    ROI = original[y-offset:y+h+offset, x-offset:x+w+offset]

    cv2.imwrite('ROI_{}.png'.format(ROI_number), ROI)
    ROI_number += 1

cv2.imshow('mask', mask)
cv2.imshow('close', close)
cv2.imshow('image', image)
cv2.waitKey()