OpenCV FREAK returns too many outliers

Question

I am trying the quite new descriptor FREAK from the latest version of OpenCV following the freak_demo.cpp example. Instead of using SURF I use FAST. My basic code is something like this:

std::vector<KeyPoint> keypointsA, keypointsB;
Mat descriptorsA, descriptorsB;
std::vector<DMatch> matches;

FREAK extractor;
BruteForceMatcher<Hamming> matcher;

FAST(imgA,keypointsA,100);
FAST(imgB,keypointsB,20);

extractor.compute( imgA, keypointsA, descriptorsA );
extractor.compute( imgB, keypointsB, descriptorsB );

matcher.match(descriptorsA, descriptorsB, matches);

• The algorithm finds a lot of matches, but there are a lot of outliers. Am I doing things right? Is there a way for tuning the algorithm?

Answer 1

When doing matching there are always some refinement steps for getting rid out of outliers.

What I usually do is discarding matches that have a distance over a threshold, for example:

for (int i = 0; i < matches.size(); i++ )
{
    if(matches[i].distance > 200)
    {
        matches.erase(matches.begin()+i-1);
    }
}

Then, I use RANSAC to see which matches fit the homography model. OpenCV has a function for this:

for( int i = 0; i < matches.size(); i++ )
    {            
        trainMatches.push_back( cv::Point2f(keypointsB[ matches[i].trainIdx ].pt.x/500.0f, keypointsB[ matches[i].trainIdx ].pt.y/500.0f) );
        queryMatches.push_back( cv::Point2f(keypointsA[ matches[i].queryIdx ].pt.x/500.0f, keypointsA[ matches[i].queryIdx ].pt.y/500.0f) );
    }

Mat h = cv::findHomography(trainMatches,queryMatches,CV_RANSAC,0.005, status);

And I just draw the inliers:

for(size_t i = 0; i < queryMatches.size(); i++) 
{
    if(status.at<char>(i) != 0) 
    {
        inliers.push_back(matches[i]);
    }
}

Mat imgMatch;
drawMatches(imgA, keypointsA, imgB, keypointsB, inliers, imgMatch);

Just try different thresholds and distances until you get the desired resutls.

Answer 2

You can also train the descriptor by giving your own selected pairs. And tune the parameters in the constructor.

explicit FREAK( bool orientationNormalized = true
       , bool scaleNormalized = true
       , float patternScale = 22.0f
       , int nbOctave = 4
       , const vector<int>& selectedPairs = vector<int>()
     );

BTW, a more efficient version of FREAK is on the way :-)