opencv: Using contours and Hough transform on rectangle detection

Question

I am trying to detect white rectangles in a grayscale image using different approaches: contour detection and Hough transform. Unfortunately there are some limitations of the image I am processing, i.e.

1. There are many features in the image and rectangle is not the only features
2. The rectangle could be merged to other features (e.g. one of the rectangle edges could be overlapped with a long straight line)
3. The rectangle could contain some other features (e.g. letter, numbers or some logo inside the rectangle)
4. There are some features look like rectangle (e.g. character 'D' looks like a rectangle with small arc on the top right and bottom right; another example is trapezoid instead of parallelogram)
5. The rectangle could be rotated from 0 to 15 degrees both clockwise and anti-clockwise
6. It is possible that the lines are broken into several lines in different lighting condition (e.g. 1 pixel gap) so the minimum line length to filter the lines has to be small (e.g. in Hough Transform)
7. When the minimum line length is set to small value, it is more often to see duplicate lines for the same line in different orientations (i.e. need to combine several lines)

For the contonours approach, the contours of some images are broken. In addition, the image could contain features like rectangle (e.g. character 'D'). I am not sure if this is a good approach.

I have seen many articles/forum suggesting to use Hough transform to detect rectangle like the following post. Unfortunately I have to set the small value of minimum line length and have seen duplicate lines. I have no idea how to deal with the points mentioned above (e.g. combine all the duplicate lines and pick only one line for each edge, how to differentiate the features with most part are lines but with small arcs like 'D', and how to isolate the square with one edge merged with a long straight line, etc).

Hough transformation vs Contour detection for Rectangle recognition with perspective projection

Any suggestions are welcomed!

EDIT: Add some pictures

Character D

Rectangle with logo and the edges are merged with long straight line

Trapezoid (with shadow on the top forming trapezoid in the bottom)

Answer 1

I would suggest that you try using a binary threshold (adaptive or otherwise) on each image, this will give some clear lines for your contour detection. You can also erode/dilate the images to remove noise (such as the thin lines in your second image)

Then use contour detection, and count the contours, finding the largest object in the image with four sides (this will probably be your object).

Make a copy of the image before you use the binary/erode, so that once you have the region of interest from contour detection, you can crop the copy image to that area.

Apologies the example links are written in python, but i'm sure once you get the idea, porting it to C++ will be easy.

Hope this helps.

EDIT

Just tried the above method myself by thresholding each image, contour detection, and then drawing a bounding box around the largest set of contours.

Below see results:

Bounding box around largest set of contours

The same, drawn over the original images