OpenCV - Use FLANN with ORB descriptors to match features

Question

I am using OpenCV 3.2

I am trying to use FLANN to match features descriptors in a faster way than brute force.

// Ratio to the second neighbor to consider a good match.
#define RATIO    0.75

void matchFeatures(const cv::Mat &query, const cv::Mat &target,
                   std::vector<cv::DMatch> &goodMatches) {
    std::vector<std::vector<cv::DMatch>> matches;
    cv::Ptr<cv::FlannBasedMatcher> matcher = cv::FlannBasedMatcher::create();
    // Find 2 best matches for each descriptor to make later the second neighbor test.
    matcher->knnMatch(query, target, matches, 2);
    // Second neighbor ratio test.
    for (unsigned int i = 0; i < matches.size(); ++i) {
        if (matches[i][0].distance < matches[i][1].distance * RATIO)
            goodMatches.push_back(matches[i][0]);
    }
}

This code is working me with SURF and SIFT descriptors, but not with ORB.

OpenCV Error: Unsupported format or combination of formats (type=0) in buildIndex

As it's said here, FLANN needs the descriptors to be of type CV_32F so we need to convert them.

if (query.type() != CV_32F) query.convertTo(query, CV_32F);
if (target.type() != CV_32F) target.convertTo(target, CV_32F);

However, this supposed fix is returning me another error in convertTo function.

OpenCV Error: Assertion failed (!fixedType() || ((Mat*)obj)->type() == mtype) in create

This assertion is in opencv/modules/core/src/matrix.cpp file, line 2277.

What's happening?

---

Code to replicate issue.

#include <opencv2/opencv.hpp>

int main(int argc, char **argv) {
    // Read both images.
    cv::Mat image1 = cv::imread(argv[1], cv::IMREAD_GRAYSCALE);
    if (image1.empty()) {
        std::cerr << "Couldn't read image in " << argv[1] << std::endl;
        return 1;
    }
    cv::Mat image2 = cv::imread(argv[2], cv::IMREAD_GRAYSCALE);
    if (image2.empty()) {
        std::cerr << "Couldn't read image in " << argv[2] << std::endl;
        return 1;
    }
    // Detect the keyPoints and compute its descriptors using ORB Detector.
    std::vector<cv::KeyPoint> keyPoints1, keyPoints2;
    cv::Mat descriptors1, descriptors2;
    cv::Ptr<cv::ORB> detector = cv::ORB::create();
    detector->detectAndCompute(image1, cv::Mat(), keyPoints1, descriptors1);
    detector->detectAndCompute(image2, cv::Mat(), keyPoints2, descriptors2);
    // Match features.
    std::vector<cv::DMatch> matches;
    matchFeatures(descriptors1, descriptors2, matches);
    // Draw matches.
    cv::Mat image_matches;
    cv::drawMatches(image1, keyPoints1, image2, keyPoints2, matches, image_matches);
    cv::imshow("Matches", image_matches);
}

Answer 1

Did you adjust the FLANN parameters?

Taken from http://docs.opencv.org/3.0-beta/doc/py_tutorials/py_feature2d/py_matcher/py_matcher.html

While using ORB, you can pass the following. The commented values are recommended as per the docs, but it didn’t provide required results in some cases. Other values worked fine.:

index_params= dict(algorithm = FLANN_INDEX_LSH, table_number = 6, # 12 key_size = 12, # 20 multi_probe_level = 1) #2

Probably you can convert that to C++ api?

According to the comment, the C++ way is:

cv::FlannBasedMatcher matcher = cv::FlannBasedMatcher(cv::makePtr<cv::flann::LshIndexParams>(12, 20, 2));