SURF and SIFT algorithms doesn't work in OpenCV 3.0 Java

Question

I am using OpenCV 3.0 (the latest version) in Java, but when I use SURF algorithm or SIFT algorithm it doesn't work and throws Exception which says: OpenCV Error: Bad argument (Specified feature detector type is not supported.) in cv::javaFeatureDetector::create

I have googled, but the answers which was given to this kind of questions did not solve my problem. If anyone knows about this problem please let me know.

Thanks in advance!

Update: The code below in third line throws exception.

        Mat img_object = Imgcodecs.imread("data/img_object.jpg");
        Mat img_scene = Imgcodecs.imread("data/img_scene.jpg");

        FeatureDetector detector = FeatureDetector.create(FeatureDetector.SURF);
        MatOfKeyPoint keypoints_object = new MatOfKeyPoint();
        MatOfKeyPoint keypoints_scene = new MatOfKeyPoint();

        detector.detect(img_object, keypoints_object);
        detector.detect(img_scene, keypoints_scene);

Answer 1

If you compile OpenCV from source, you can fix the missing bindings by editing opencv/modules/features2d/misc/java/src/cpp/features2d_manual.hpp yourself.

I fixed it by making the following changes:

(line 6)
#ifdef HAVE_OPENCV_FEATURES2D
#include "opencv2/features2d.hpp"
#include "opencv2/xfeatures2d.hpp"
#include "features2d_converters.hpp"

...(line 121)
    case SIFT:
    fd = xfeatures2d::SIFT::create();
    break;
    case SURF:
    fd = xfeatures2d::SURF::create();
    break;

...(line 353)
    case SIFT:
        de = xfeatures2d::SIFT::create();
        break;
    case SURF:
        de = xfeatures2d::SURF::create();
        break;

The only requirement is that you build opencv_contrib optional module along with your sources (you can download the git project from https://github.com/Itseez/opencv_contrib and just set its local path on opencv's ccmake settings.

Oh, and keep in mind that SIFT and SURF are non-free software ^^;

Answer 2

That is because they are not free in newer versions of OpenCV (3+). I faced that problem some time ago. You have to:

1. Download OpenCV (if you have not)
2. Download the nonfree part from opencv github repo
3. Generate the makefiles with cmake -DBUILD_SHARED_LIBS=OFF specifying the nonfree part with DOPENCV_EXTRA_MODULES_PATH=../opencv_contrib/modules option and build with make -j8 (or whatever Java version you use)
4. Edit features2d_manual.hpp file, including opencv2/xfeatures2d.hpp and including the necessary code for SIFT and SURF case, which are commented and not defined: fd=xfeatures2d::SIFT::create(); for SIFT descriptor and de = xfeatures2d::SIFT::create(); for SIFT extractor. Do the same for SURF if you want to use it too.

I wrote this post explaining step by step how to compile the non-free OpenCV part in order to use privative tools like SIFT or SURF. Compile OpenCV non-free part.

Answer 3

I believe changing features2d module (FeatureDetector class or any other classes from features2d_manual.hpp) to enable methods from OpenCV contrib modules is less attractive approach because it leads to circular dependency between the "core" OpenCV and extensions (which can be non-free or experimental). There is another way to fix this issue without affecting feature2d classes. Making changes in xfeatures2d CMakeLists.txt as described here leads to generation of java wrappers for SIFT and SURF - opencv-310.jar has org.opencv.xfeatures2d package now. Some fix was required in /opencv/modules/java/generator/gen_java.py. Namely inserted 2 lines as shown below:

def addImports(self, ctype):
if ctype.startswith('vector_vector'):
    self.imports.add("org.opencv.core.Mat")
    self.imports.add("org.opencv.utils.Converters")
    self.imports.add("java.util.List")
    self.imports.add("java.util.ArrayList")
    self.addImports(ctype.replace('vector_vector', 'vector'))
elif ctype.startswith('Feature2D'):                                           #added
    self.imports.add("org.opencv.features2d.Feature2D")                       #added
elif ctype.startswith('vector'):
    self.imports.add("org.opencv.core.Mat")
    self.imports.add('java.util.ArrayList')
    if type_dict[ctype]['j_type'].startswith('MatOf'):
        self.imports.add("org.opencv.core." + type_dict[ctype]['j_type'])
    else:
        self.imports.add("java.util.List")
        self.imports.add("org.opencv.utils.Converters")
        self.addImports(ctype.replace('vector_', ''))

After these changes wrappers are generated successfully. However the main problem still remains, how to use these wrappers from Java )). For example SIFT.create() gives the pointer to a new SIFT class but calling any class method (for instance detect()) crashes Java. I also noticed that using MSER.create() directly from Java leads to the same crash.

So it looks like the problem is isolated to the way how Feature2D.create() methods are wrapped in Java. The solution seems to be the following (again, changing /opencv/modules/java/generator/gen_java.py):

Find the string:

ret = "%(ctype)s* curval = new %(ctype)s(_retval_);return (jlong)curval->get();" % { 'ctype':fi.ctype }

Replace it with the following:

ret = "%(ctype)s* curval = new %(ctype)s(_retval_);return (jlong)curval;" % { 'ctype':fi.ctype }

Rebuild the opencv. That is it, all create() methods will start working properly for all children of Feature2D class including experimental and non-free methods. FeatureDescriptor/DescriptorExtractor wrappers can be deprecated I think as Feature2D is much easier to use.

BUT! I'm not sure if the suggested fix is safe for other OpenCV modules. Is there a scenario when (jlong)curval needs to be dereferenced? It looks like the same fix was suggested already here.