How to train and predict using bag of words?

Question

I have a folder of images of a car from every angle. I want to use the bag of words approach to train the system in recognizing the car. Once the training is done, I want that if an image of that car is given it should be able to recognize it.

I have been trying to learn the BOW function in opencv in order to make this work and have come at a level where I do not know what to do now and some guidance would be appreciated.

Here is my code that I used to make the bag of words:

Ptr<FeatureDetector> features = FeatureDetector::create("SIFT");
    Ptr<DescriptorExtractor> descriptors = DescriptorExtractor::create("SIFT");
    Ptr<DescriptorMatcher> matcher = DescriptorMatcher::create("FlannBased");

    //defining terms for bowkmeans trainer
    TermCriteria tc(MAX_ITER + EPS, 10, 0.001);
    int dictionarySize = 1000;
    int retries = 1;
    int flags = KMEANS_PP_CENTERS;
    BOWKMeansTrainer bowTrainer(dictionarySize, tc, retries, flags);

    BOWImgDescriptorExtractor bowDE(descriptors, matcher);

    //training data now
    Mat features;
    Mat img = imread("c:\\1.jpg", 0);
    Mat img2 = imread("c:\\2.jpg", 0);
    vector<KeyPoint> keypoints, keypoints2;
    features->detect(img, keypoints);
    features->detect(img2,keypoints2);
    descriptor->compute(img, keypoints, features);
    Mat features2;
    descripto->compute(img2, keypoints2, features2);
    bowTrainer.add(features);
    bowTrainer.add(features2);

    Mat dictionary = bowTrainer.cluster();
    bowDE.setVocabulary(dictionary);

This is all based on the BOW documentation.

I think at this stage my system is trained. and the next step is predicting.

this is where I dont know what to do. If I use SVM or NormalBayesClassifier they both use the terms train and predict.

How do I predict and train after this? any guidance would be much appreciated. How do I connect the training of the classifier to my `bowDE`` function?

Answer 1

Your next step is to extract the actual bag of word descriptors. You can do this using the compute function from the BOWImgDescriptorExtractor. Something like

 bowDE.compute(img, keypoints, bow_descriptor);

Using this function you create descriptors which you then gather into a matrix which serves as the input for the classifier functions. Maybe this tutorial can guide you a little bit.

Another thing I would like to mention is, that for classification you usually need at least 2 classes. So you also need some images which do not contain cars to train a classifier.