Parts Recognition / Classification with OpenCV

Question

I am just starting to explore OpenCV and the EmguCV .NET wrappers for it and need some general direction from folks who understand the big picture of its capabilities and perhaps those who have tackled a task similar to what I need to accomplish.

I will have a series of still photos and in each image will appear an object or nothing. The objects are pieces of metal hardware (bolts) and will be laying on their side with their length parallel to the top/bottom of the image (i.e., the picture is taken from above). If there is an object, it will be one of about 100 discrete types of bolt, some with very similar though not identical features and dimensions. For example, they will all be mostly rectangular in profile but will vary in length and width (diameter) and can have heads that are either hexagonal or round (which will be seen in profile as rectangular or as the minor segment of a circle, respectively) or will have conical heads for countersunk applications. An illustration of the types of parts I'm talking about (this is just to show the types of parts - my images are photos of single parts):

_{(source: donsnotes.com)}

I need to classify them such that all sizes and types are differentiated. A 1-3/8" bolt should classify as different from a 1-1/2" bolt even if they are the same diameter and have the same head type. The minimum length difference between sizes will be 1/8", not the standard 1/16".

If it matters, I have good control of the following:

• Lighting (but back-lighting isn't going to be practical)
• The appearance of the background (useful for background subtraction maybe?)
• The distance from the camera to the object (identical objects will always appear to be the same size in the images)
• Generally, the position of the bolt - it will laying horizontally on its side, parallel with the top/bottom edges of the image frame. I cannot control whether its head is to the left or right in the image.

Unfortunately I cannot find any online papers or articles that directly address what I need to do - but lots that illustrate simpler tasks like finding a colored ball or finding rectangles. I can't find anything on identifying and classifying each of a large(ish) number of different but similar shapes. I do have two of the suggested OpenCV books and while they are great, they don't seem to address this issue.

I have found pretty clean Canny edges on my sample images but there is lots of noise in the interior of the part due to lighting. This makes finding clean Hough line segments kind of spotty.

I'm not certain if I should try to pare down the list of possible matches using absolute dimensions calculated by measuring across the Canny edges - then use something more robust like a cascade classifier...? Or what.

I'm really just looking for someone's opinion on a general strategy or a point in the right direction...

Can anyone give me something to start trying? I'm really at a loss.

Thanks!

Answer 1

From my experience, I would suggest you the following. You know better than most of us how to classify manually, right? So think what happens in your brain while you classify. Suppose, you see a circular shape from the top view, so now your problem becomes how to identify that shape? Post such problems here, it will be lot easier for other people. Feature extraction is nothing but what brain thinks when you see that object.

For this question, I would suggest you first keep a dark background so that it will become necessary to binarize that image. Then if its a top view, you can see either a circle, hexagon etc. Then get only edges. Then get a minimal bounding circle and get its diameter. Algorithm and code to find minimum bounding circle can be obtained here.

As far as shape is concerned, I suggest you take gradient of that binary image and then calculate the angle of that gradients only at the edge points (which you got in top view). Histogram of that gradients will be your feature vector. See the bar plot of different shapes. If your brain can distinguish, then you can think about which classifier to use. I would rather not comment on that now since it depends on lot of things, such as distribution of features, their separability and most importantly speed requirement. But don't worry about classifier now.

Now, lets try to tackle the height. I assume you can get the front view, then just calculate the bounding box (you can get that from regionprops function in MATLAB).

Note that I am just imagining and talking all these things. You have to first do what I said in the first paragraph and then see if other part of the answer makes sense to you. I assume you have certain mathematical background to understand certain terminologies in this answer. If not, please do not hesitate to ask.

P.S. +1 for a good question.