Rectangle detection inaccuracy using approxPolyDP() in openCV

Question

As part of a program which contains a series of images to be processed, I first need to detect a green-coloured rectangle. I'm trying to write a program that doesn't use colour masking, since the lighting and glare on the images will make it difficult to find the appropriate HSV ranges.

(p.s. I already have two questions based on this program, but this one is unrelated to those. It's not a follow up, I want to address a separate issue.)

I used the standard rectangle detection technique, making use of findContours() and approxPolyDp() methods. I added some constraints that got rid of unnecessary rectangles (like aspectRatio>2.5, since my desired rectangle is clearly the "widest" and area>1500, to discard random small rectangles) .

import numpy as np
import cv2 as cv

img = cv.imread("t19.jpeg")

width=0 
height=0

start_x=0 
start_y=0
end_x=0 
end_y=0


output = img.copy()
gray = cv.cvtColor(img, cv.COLOR_BGR2GRAY)


#threshold
th = cv.adaptiveThreshold(gray,255,cv.ADAPTIVE_THRESH_GAUSSIAN_C,cv.THRESH_BINARY,9,2)

cv.imshow("th",th)



#rectangle detection

contours, _ = cv.findContours(th, cv.RETR_TREE, cv.CHAIN_APPROX_NONE)

for contour in contours:

    approx = cv.approxPolyDP(contour, 0.01* cv.arcLength(contour, True), True)
    
    cv.drawContours(img, [approx], 0, (0, 0, 0), 5)
    
    x = approx.ravel()[0]
    y = approx.ravel()[1]

    x1 ,y1, w, h = cv.boundingRect(approx)
    a=w*h    
    if len(approx) == 4 and x>15  :
            
        aspectRatio = float(w)/h
        if  aspectRatio >= 2.5 and a>1500:          
          print(x1,y1,w,h)
          width=w
          height=h   
          start_x=x1
          start_y=y1
          end_x=start_x+width
          end_y=start_y+height      
          cv.rectangle(output, (start_x,start_y), (end_x,end_y), (0,0,255),3)
          cv.putText(output, "rectangle "+str(x1)+" , " +str(y1-5), (x1, y1-5), cv.FONT_HERSHEY_COMPLEX, 0.5, (0, 0, 0))
          
cv.imshow("op",output)

print("start",start_x,start_y)
print("end", end_x,end_y)
print("width",width)
print("height",height)

It is working flawlessly for all the images, except one:

enter image description here

I used adaptive thresholding to create the threshold, which was used by the findContours() method. I tried displaying the threshold and the output , and it looks like this:

enter image description here

The thresholds for the other images also looked similar...so I can't pinpoint what exactly has gone wrong in the rectangle detection procedure.

Some tweaks I have tried:

1. Changing the last two parameters in the adaptive parameters method. I tried 11,1 , 9,1, and for both of them, the rectangle in the threshold looked more prominent : but in this case the output detected no rectangles at all.
2. I have already disregarded otsu thresholding, as it is not working for about 4 of my test images.

What exactly can I tweak in the rectangle detection procedure for it to detect this rectangle?

I also request , if possible, only slight modifications to this method and not some entirely new method. As I have mentioned, this method is working perfectly for all of my other test images, and if the new suggested method works for this image but fails for the others, then I'll find myself back here asking why it failed.

Edit: The method that abss suggested worked for this image, however failed for:

• image 4
• image 1, far off

Other test images:

• image 1, normal
• image 2
• image 3
• image 9, part 1
• image 9, part 2

Answer 1

You can easily do it by adding this line of code after your threshold

kernel = cv.getStructuringElement(cv.MORPH_RECT,(3,3))
th = cv.morphologyEx(th,cv.MORPH_OPEN,kernel)

This will remove noise within the image. you can see this link for more understanding about morphologyEx https://docs.opencv.org/master/d9/d61/tutorial_py_morphological_ops.html

The results I got is shown below

Answer 2

I have made a few modifications to your code so that it works with all of your test images. There are a few false positives that you may have to filter based on HSV color range for green (since your target is always a shade of green). Alternately you can take into account the fact that the one of the child hierarchy of your ROI contour is going to be > 0.4 or so times than the outer contour. Here are the modifications:

1. Used DoG for thresholding useful contours
2. Changed arcLength multiplier to 0.5 instead of 0.1 as square corners are not smooth
3. cv2.RETR_CCOMP to get 2 level hierarchy
4. Moved ApproxPolyDP inside to make it more efficient
5. Contour filter area changed to 600 to filter ROI for all test images
6. Removed a little bit of unnecessary code

Check with all the other test images that you may have and modify the parameters accordingly.

img = cv2.imread("/path/to/your_image")

width=0 
height=0

start_x=0 
start_y=0
end_x=0 
end_y=0

output = img.copy()
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

gw, gs, gw1, gs1, gw2, gs2 = (3,1.0,7,3.0, 3, 2.0)

img_blur = cv2.GaussianBlur(gray, (gw, gw), gs)
g1 = cv2.GaussianBlur(img_blur, (gw1, gw1), gs1)
g2 = cv2.GaussianBlur(img_blur, (gw2, gw2), gs2)
ret, thg = cv2.threshold(g2-g1, 127, 255, cv2.THRESH_BINARY)

contours, hier = cv2.findContours(thg, cv2.RETR_CCOMP, cv2.CHAIN_APPROX_NONE)

img_cpy = img.copy()

width=0 
height=0

start_x=0 
start_y=0
end_x=0 
end_y=0

for i in range(len(contours)):
    
    if hier[0][i][2] == -1:
        continue
        
    x ,y, w, h = cv2.boundingRect(contours[i])
    a=w*h    
    aspectRatio = float(w)/h
    if  aspectRatio >= 2.5 and a>600:          
        approx = cv2.approxPolyDP(contours[i], 0.05* cv2.arcLength(contours[i], True), True)
        if len(approx) == 4 and x>15  :
            width=w
            height=h   
            start_x=x
            start_y=y
            end_x=start_x+width
            end_y=start_y+height      
            cv2.rectangle(img_cpy, (start_x,start_y), (end_x,end_y), (0,0,255),3)
            cv2.putText(img_cpy, "rectangle "+str(x)+" , " +str(y-5), (x, y-5), cv2.FONT_HERSHEY_COMPLEX, 0.5, (0, 0, 0))
          
plt.imshow(img_cpy)

print("start",start_x,start_y)
print("end", end_x,end_y)