What is the simplest *correct* method to detect rectangles in an image?

Question

I am trying to think of the best method to detect rectangles in an image.

My initial thought is to use the Hough transform for lines, and to select combinations of lines where you have two lines intersected at both the lower portion and upper portion by the same two lines, but this is not sufficient.

Would using a corner detector along with the Hough transform work?

Answer 1

Check out /samples/c/squares.c in your OpenCV distribution. This example provides a square detector, and it should be a pretty good start.

My answer here also applies.

Answer 2

I don't think that currently there exists a simple and robust method to detect rectangles in an image. You have to deal with many problems such as the rectangles not being exactly rectangular but only approximately, partial occlusions, lighting changes, etc.

One possible direction is to do a segmentation of the image and then check how close each segment is to being a rectangle. Since you can't trust your segmentation algorithm, you can run it multiple times with different parameters.

Another direction is to try to parametrically fit a rectangle to the image such that the image gradient magnitude along the contour will be maximized.

If you choose to work on a parametric approach, notice that while the trivial way to parameterize a rectangle is by the locations of it's four corners, which is 8 parameters, there are a few other representations that require less parameters.