detect whether the checkbox is checked using opencv

Question

i am trying to find if a checkbox([])1

is checked or not. till now i tried using the method used in pyimagesearch.i am trying to see if there are any triangles in the image. if there is at least one triangle it means that the checkbox is checked. The problem i found is that, sometimes the outer rectangle is being detected instead of triangles. how should i ignore rectangle and detect only triangles

Answer 1

I think you don't even need to use hierarchy information, Just count the number of contours and use to classify your image. I am giving a simple python implementation to view the individual contours.

Contour hierarchy representation in OpenCV has been explained here Contour Hierachy

img = cv2.imread(r'D:/Image/cross.png')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
ret,bw = cv2.threshold(gray,220,255,cv2.THRESH_BINARY_INV)
_,contours,hierarchy = cv2.findContours(bw, cv2.RETR_CCOMP,1)
cntLen = 10
ct = 0 #number of contours
for cnt in contours:
    if len(cnt) > cntLen: #eliminate the noises
        ct += 1
        newimg = img.copy()
        cv2.drawContours(newimg,[cnt],0,(0,0,255),2)
        cv2.imshow('Win', newimg)
        cv2.waitKey(0)
print('Total contours: ',ct)

Answer 2

I ended up needing to do something very similar to this and had to determine whether or not 2,500-ish envelopes had a check on them. Using the answer here, I was able to do it with about a 10% error rate. I ended up doing the following, instead:

First I fed my 'Unchecked' samples through a PixelDensity script. I'm checking for the number of 'white' pixels:

import cv2
import numpy as np
from pathlib import Path
from os import listdir
import functools

whitePixelAverage = []

path = "./"
included_extensions = ['png','PNG']

allFiles = [f for f in listdir(path) if any(f.endswith(ext) for ext in included_extensions)] # Get all files in current directory

length = len(allFiles)

for i in range(length):
    img = cv2.imread(allFiles[i], cv2.IMREAD_GRAYSCALE)
    imgCrop = img[655:713,692:767] # Select your ROI here, I allow for a bit of deviation between image scans
    n_white_pix = np.sum(imgCrop == 255)
    print('number of white pixels: ', n_white_pix)
    whitePixelAverage.append(n_white_pix)
print(whitePixelAverage)
print("Average: ", functools.reduce(lambda x, y: x + y,whitePixelAverage)/len(whitePixelAverage))
print("Lowest: ", min(whitePixelAverage))
print("highest: ", max(whitePixelAverage))

t = input(":") # Wait for input to close

From this selection, I ended up taking my 'Lowest' value and giving it a Threshold of 3% (in my case this was about 117).

Now, I do the same, but this time feed it a sample of 'Checked' boxes. The White Pixel density of these should be, on average, lower.

For my checked boxes, I use the Average. I end up comparing the Average of the Checked boxes to the lowest+threshold of the unchecked:

from imutils.perspective import four_point_transform
from imutils import contours
import numpy as np
import imutils
import cv2
from pathlib import Path
from os import listdir
import os
import shutil
import functools

path = "./testSet/"
checkedP = "./testSet/checked/"
uncheckedP = "./testSet/unchecked/"
included_extensions = ['png','PNG']

THRESHOLD = 117

allFiles = [f for f in listdir(path) if any(f.endswith(ext) for ext in included_extensions)] # Get all files in current directory

length = len(allFiles)

for i in range(length):
    img = cv2.imread(path+allFiles[i])
    imgCrop = img[655:713,692:767]
    gray = cv2.cvtColor(imgCrop, cv2.COLOR_BGR2GRAY)
    ret,bw = cv2.threshold(gray,220,255,cv2.THRESH_BINARY_INV)
    _,contours,hierarchy = cv2.findContours(bw, cv2.RETR_CCOMP,1)
    cntLen = 1
    ct = 0 #number of contours
    for cnt in contours:
        if len(cnt) > cntLen: #eliminate the noises
            ct += 1
    print('Total contours: ',ct)
    if ct >= 3:
        print("Checked!")
        shutil.move(path + allFiles[i], checkedP + allFiles[i])
        img = cv2.imread(checkedP+allFiles[i], cv2.IMREAD_GRAYSCALE)
        imgCrop = img[655:713,692:767]
        n_white_pix = np.sum(imgCrop == 255)
        if (n_white_pix > 3747+THRESHOLD):
            shutil.move(checkedP + allFiles[i], checkedP+"Unconfident/")
        averageWhitePixels.append(n_white_pix)
    else:
        print("Not so checked")
        shutil.move(path + allFiles[i], uncheckedP + allFiles[i])
        img = cv2.imread(uncheckedP+allFiles[i], cv2.IMREAD_GRAYSCALE)
        imgCrop = img[655:713,692:767]
        n_white_pix = np.sum(imgCrop == 255)
        if (n_white_pix < 3747+THRESHOLD):
            shutil.move(uncheckedP + allFiles[i], uncheckedP+"Unconfident/")

Comparing Pixel Density in addition to flamelite's answer, I was able to reduce my error rate from 10% to about 2.47% over 2,500 different samples.

I apologize for how inelegant my code is. This is my first answer, please let me know if anything is too terrible. There are most likely lots of things left in there that I used for troubleshooting.

Hope this helps!