Creating OpenCV Haar Classifier from an existing model

Question

I want to make an application that recognizes screws. there is a standard for each screw size and shape, and there are softwares that provide a 360 3d model of each screw (Autocad for example)

I wanted to know if there is a any way to take an existing model (of some extension) and create an Haar classifier from it.

If it wasn't clear, I don't want to take thousands of pictures of every model. I want to somehow reuse the existing models

Thanks

Answer 1

In short: I don't think it is possible, unless you are able to render your 3D models realistically according to various lighting conditions.

Haar and LBP classifiers do use some kind of texture information, so the pure shape information isn't enough, afaik. (so instead of HAAR, HoG might or might not be more appropriate for such an approach)

For haar classifier training you could render your 3D models with different material and lighting conditions and in various poses. You'll need background images however as negative samples and as background for your rendered screws (but you could merge the screws to background with opencv_createsamples tool afterwards, too).

So depending on how realistically or appropriate you are able to render the screews, you can use them for your training. Keep in mind that in reality, two screws of same type don't look perfectly the same, the length might vary in small amounts (depending on production quality) and there might be deformations or "noise" like burrs might be present that differ from the perfect model, maybe you would have to consider that in your rendering.

In addition, keep in mind, that haar and lbp cascade classifier work best if they cover a class only from a single pose (like frontal faces or profile faces, not both at the same time), this reduces the use of a signle 3D model.

sometimes, 3D models can be used in combination with chamfer matching to compare them to 2D models.

Answer 2

A Haar classifier operates on 2D images, both for training and recognition. It requires a set of normalized positive 2D images (with screw) and a set of normalized negative 2D images (without screw) for training the classifier. It thus cannot be trained directly with a 3D model, if that is your question. Of course you could render 2D images using the 3D model, but as I understood your comment at the end of your question, this is not what you would like to do.

Besides, depending on your actual task, a Haar classifier is unlikely to be the method of choice. Some important questions are: Can you segment the screw in the image? Can there be several screws in the image? Is it a classification task or do you know which screw type is present in the image? Do you require localization or is it only about recognition? Are the possible poses of your screws restricted (or can be normalized beforehand due to segmentation)?

Note that a Haar classifier is meant for "learning" variations of the appearance of different instances of a class with similar pose (like more or less upright faces of different persons, i.e. face detection) and not for handling completely different poses of the same object, not even in 2D, not to even talk about 3D. So it could be only applicable in a very restricted scenario.

UPDATE: A few hints to lead you into the right direction:

1. If you can segment the screws and want to try a simple approach, try to treat it as a 2D problem (assuming the screws do not stand upright) and perform normalization on the basis of image moments.

2. If you can segment the screws and want to treat it as a 2D/3D problem (a 3D model projected to a 2D image), you can look into my (now old) research on efficient appearance-based matching against views generated with a 3D model.

3. If you cannot segment the screw and you know which type of screw is present in the image, you can look into so-called geometric pattern matching from an industrial machine vision library, e.g. Shape-based matching from Halcon. It offers training from CAD-like data as well.

4. If you cannot segment the screw and you don't know which type of screw is present in the image, you can train a Deep Convolutional Neural Network for the classification task and then use 3. for verification and localization.