Performance issues using BRISK detector/descriptor OpenCV

Question

I'm experiencing a performance problem when I use BRISK for feature detection and description in OpenCV.

Basically I try to match all the descriptors I get from this image:

against all the descriptors I get from a database of images using flann based matcher with the LSH algorithm and BRISK for feature detection and description.

My database of images is composed of 242 images. In this 242 images there are three images corresponding to each object taken separately in the above "complex" image query.

Here are the parameters used for BRISK detection (default opencv parameters): Treshold: 30, Octaves: 4, Pattern scale: 1.0.

After the flann matching using the best match technic (Each descriptor in the image query is associated with its nearest neighborhood in the set of database descriptors), my algorithm output the list of database images sorted by percentage of matching. Here are the four first results:

1. Image corresponding to the bolt in the database: 4 matches / 15 detected keypoints => Matching percentage: 26.7%.
2. Image corresponding to the bottle in the database has 4 matches / 15 detected keypoints => Matching percentage: 26.7%.
3. Image corresponding to the bolt in the database has 10 matches / 59 detected keypoints => Matching percentage: 16.9%.
4. Image corresponding to an object which is not present in the image query: 1 match / 16 detected keypoints => Matching percentage: 16.7%.

I compared this results using ORB as feature detection and description. Here are the parameters used: Number of features: 2000, Scale factor: 1.2, Number of pyramid levels: 8.

Here are the results I get:

1. Image corresponding to the bolt in the database: 576 matches / 752 detected keypoints => Matching percentage: 76.6%.
2. Image corresponding to the bottle in the database has 111 matches / 189 detected keypoints => Matching percentage: 58.7%.
3. Image corresponding to the pen in the database has 124 matches / 293 detected keypoints => Matching percentage: 42.3%.
4. Image corresponding to an object which is not present in the image query: 2 matches / 66 detected keypoints => Matching percentage: 3%.

As you can see the results are far much better with ORB. Firstly there are more keypoints detected on each image in the database and the percentage of matching is significantly better for the good objects. Moreover the gap between the percentage of matching for the good objects and the percentage of matching for the wrong objects is more significant.

I am wondering why BRISK detector detects much less keypoints than the ORB detector. I have made different tests to figure out how I could detects more keypoints with the BRISK detector (Decrease the threshold, decrease the octave number). I can indeed detect a little more keypoints but the difference with the ORB detector is still really important. Do you have any idea why BRISK detector has a such behavior?

My OpenCV version is 2.4.8 but I have tried the BRISK detection part with the 2.4.4 and the 2.4.9 version according to these statements:

http://code.opencv.org/issues/2491 and BRISK feature detector detects zero keypoints without improvements.

I have also tried to combine the ORB detector with the BRISK description. Matching results are better than with the first method (full BRISK) but worse than the second one (full ORB):

1. Image corresponding to the bolt in the database: 529 matches / 708 detected keypoints => Matching percentage: 74.7%.
2. Image corresponding to the bottle in the database has 69 matches / 134 detected keypoints => Matching percentage: 51.5%.
3. Image corresponding to the pen in the database has matches 93 / 247 detected keypoints => Matching percentage: 37.6%.
4. Image corresponding to an object which is not present in the image query: 5 matches / 50 detected keypoints => Matching percentage: 10%.

Note that the number of keypoints detected on each image is not the same in method 2 and method 3. Actually when I run this code on a test image (here the image of the bolt):

// BRISK parameters
int Threshl=30;
int Octaves=4;
float PatternScales=1.0f;

// ORB parameters
int nFeatures=2000;
float scaleFactor=1.2f;
int nLevels=8;


BRISK  BRISKD(Threshl, Octaves, PatternScales);
ORB ORBD(nFeatures, scaleFactor, nLevels);

vector<KeyPoint> kpts;
Mat descriptors;

Mat img = cv::imread("Path to the bolt image", IMREAD_GRAYSCALE );
ORBD.detect(img,kpts); // Number of keypoints detected = 752
BRISKD.compute(img, kpts, descriptors); // Number of descriptors = 708

BRISK seems not to compute all the detected keypoints in descriptors (752 keypoints detected by ORB detector => 708 descriptors computed by BRISK descriptor).

However, even if all the keypoints detected by ORB are not computed in BRISK descriptors. It seems with these results that BRISK is not better to describe a keypoint than ORB? Correct me if I'm wrong but I'm pretty sure it should be the contrary...

If you guys have any element which could help me understand my results or if you have already experienced some problems with BRISK in OpenCV please let me know. Any help would be greatly appreciated :).

Answer 1

Ok I figured out how to get the results I expected from BRISK. I tried the code from the BRISK creator here: https://github.com/calihem/mavhub/tree/master/thirdparty

Here are the results I get:

1. Image corresponding to the bolt in the database: 149 matches / 288 detected keypoints => Matching percentage: 51.7%.
2. Image corresponding to the bottle in the database has 27 matches / 57 detected keypoints => Matching percentage: 47.4%.
3. Image corresponding to the pen in the database has 38 matches / 101 detected keypoints => Matching percentage: 37.6%.
4. Image corresponding to an object which is not present in the image query: 5 matches / 76 detected keypoints => Matching percentage: 6.6%.

These results are much better than those I had before. Actually they are not better than those obtained with ORB for this specific query image. However I tried whith other query images, specially more complex images and BRISK outperforms ORB.

It seems there is a problem with BRISK implementation in opencv at least from the 2.4.4 version till the current version 2.4.9. I think this bug should be fixed soon because it has been reported to the opencv community (http://code.opencv.org/issues/2491).

In the meantime I recommend you to use the code written by the BRISK creator which works well ;).