Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Algorithm to detect corners of paper sheet in photo

What is the best way to detect the corners of an invoice/receipt/sheet-of-paper in a photo? This is to be used for subsequent perspective correction, before OCR.

My current approach has been:

RGB > Gray > Canny Edge Detection with thresholding > Dilate(1) > Remove small objects(6) > clear boarder objects > pick larges blog based on Convex Area. > [corner detection - Not implemented]

I can't help but think there must be a more robust 'intelligent'/statistical approach to handle this type of segmentation. I don't have a lot of training examples, but I could probably get 100 images together.

Broader context:

I'm using matlab to prototype, and planning to implement the system in OpenCV and Tesserect-OCR. This is the first of a number of image processing problems I need to solve for this specific application. So I'm looking to roll my own solution and re-familiarize myself with image processing algorithms.

Here are some sample image that I'd like the algorithm to handle: If you'd like to take up the challenge the large images are at http://madteckhead.com/tmp

case 1
(source: madteckhead.com)

case 2
(source: madteckhead.com)

case 3
(source: madteckhead.com)

case 4
(source: madteckhead.com)

In the best case this gives:

case 1 - canny
(source: madteckhead.com)

case 1 - post canny
(source: madteckhead.com)

case 1 - largest blog
(source: madteckhead.com)

However it fails easily on other cases:

case 2 - canny
(source: madteckhead.com)

case 2 - post canny
(source: madteckhead.com)

case 2 - largest blog
(source: madteckhead.com)

Thanks in advance for all the great ideas! I love SO!

EDIT: Hough Transform Progress

Q: What algorithm would cluster the hough lines to find corners? Following advice from answers I was able to use the Hough Transform, pick lines, and filter them. My current approach is rather crude. I've made the assumption the invoice will always be less than 15deg out of alignment with the image. I end up with reasonable results for lines if this is the case (see below). But am not entirely sure of a suitable algorithm to cluster the lines (or vote) to extrapolate for the corners. The Hough lines are not continuous. And in the noisy images, there can be parallel lines so some form or distance from line origin metrics are required. Any ideas?

case 1case 2 case 3 case 4
(source: madteckhead.com)

like image 422
Nathan Keller Avatar asked Jul 02 '11 07:07

Nathan Keller


People also ask

What is corner detection in image processing?

Corner detection works on the principle that if you place a small window over an image, if that window is placed on a corner then if it is moved in any direction there will be a large change in intensity.

How can you detect corner of images using OpenCV?

OpenCV has algorithms available that can allow us to detect corners in an image. There are 2 main algorithms used in OpenCV for corner detection: the Harris corner detection method and the goodFeaturesToTrack() method.


1 Answers

I'm Martin's friend who was working on this earlier this year. This was my first ever coding project, and kinda ended in a bit of a rush, so the code needs some errr...decoding... I'll give a few tips from what I've seen you doing already, and then sort my code on my day off tomorrow.

First tip, OpenCV and python are awesome, move to them as soon as possible. :D

Instead of removing small objects and or noise, lower the canny restraints, so it accepts more edges, and then find the largest closed contour (in OpenCV use findcontour() with some simple parameters, I think I used CV_RETR_LIST). might still struggle when it's on a white piece of paper, but was definitely providing best results.

For the Houghline2() Transform, try with the CV_HOUGH_STANDARD as opposed to the CV_HOUGH_PROBABILISTIC, it'll give rho and theta values, defining the line in polar coordinates, and then you can group the lines within a certain tolerance to those.

My grouping worked as a look up table, for each line outputted from the hough transform it would give a rho and theta pair. If these values were within, say 5% of a pair of values in the table, they were discarded, if they were outside that 5%, a new entry was added to the table.

You can then do analysis of parallel lines or distance between lines much more easily.

Hope this helps.

like image 111
Daniel Crowley Avatar answered Sep 18 '22 12:09

Daniel Crowley