OpenCV: Detect squares in dark background

Question

currently I am trying to calculate optical flows of moving objects. the objects in particular are the squares that are around the circular knobs:

Here is the vanilla image I am trying to process:

my concern is about the right bottom-most strip. The two squares are usually unable to be detected when I have tried Canny Edge detection or GoodFeaturesToTrack. I am currently trying a sharpen kernel and a threshold, then morphological transformation to finding the contour areas. However, when I threshold I get the following results:

import numpy as np
import cv2 as cv
from matplotlib import pyplot as plt


filename = 'images/Test21_1.tif'


image = cv.imread(filename)

kernel = [ [0, -1, 0], [-1, 5, -1], [0, -1, 0] ] #sharpen kernel I got from wikipedia

kernel = np.array(kernel)
dst = cv.filter2D(image, -1, kernel)
ret, thresh = cv.threshold(dst, 80, 150, cv.THRESH_BINARY_INV)

plt.subplot(121),plt.imshow(image),plt.title('Original')
plt.xticks([]), plt.yticks([])
plt.subplot(122),plt.imshow(thresh),plt.title('Threshold')
plt.xticks([]), plt.yticks([])
plt.show()

I was wondering what I could do in openCV to be able to recognize that square. These squares are the objects that move in the videos, and I wish to use them to calculate their optical flow. I am currently considering resorting to a PyTorch CNN to detect the features. I would manually label the images for training/test datasets, but I believe that may be a bit overkill. Thank you for your time.

Answer 1

I am not sure if this is any better, but you can try using the division normalization technique in Python/OpenCV.

• Read the input
• Convert to grayscale
• Apply morphology
• Divide the input by the result from the morphology
• Adaptive Threshold
• Save the results

import cv2
import numpy as np

# read the image
img = cv2.imread('rods.png')

# convert to gray
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)

# apply morphology
kernel = cv2.getStructuringElement(cv2.MORPH_RECT , (5,5))
smooth = cv2.morphologyEx(gray, cv2.MORPH_DILATE, kernel)

# divide gray by morphology image
division = cv2.divide(gray, smooth, scale=255)

# threshold
thresh = cv2.adaptiveThreshold(gray, 255, cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY, 7, 4)

# save results
cv2.imwrite('rods.division.jpg',division)
cv2.imwrite('rods.thresh.jpg',thresh)

# show results
cv2.imshow('smooth', smooth)  
cv2.imshow('division', division)  
cv2.imshow('thresh', thresh)  
cv2.waitKey(0)
cv2.destroyAllWindows()

Division image:

Threshold image:

Answer 2

The problem is local contrast is bad near the bottom right square. Can you try by using CLAHE (Contrast Limited Adaptive Histogram Equalization).

# improving local contrast
GRID_SIZE = 20
clahe = cv2.createCLAHE(clipLimit=2.0, tileGridSize=(GRID_SIZE,GRID_SIZE))
image = clahe.apply(image)

Then try using your algorithm to detect the squares.