Why does Python implementation and Java implementation of OpenCV's MSER create different output?

Question

I have been trying to use both the Python implementation (opencv 2.4.11) and the Java implementation (opencv 2.4.10) of OpenCV's MSER algorithm. Interestingly, I've noticed that MSER's detect returns different types of output in Python vs Java. In Python, detect returns a list of lists of points, where each list of points represents a blob detected. In Java, a Mat is returned, where each row is a single point with an associated diameter representing a blob detected. I would like to reproduce the Python behavior in Java, where blobs are defined by a set of points, not one point. Anyone know what's going on?

Python:

frame = cv2.imread('test.jpg')
mser = cv2.MSER(**dict((k, kw[k]) for k in MSER_KEYS))  
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)  
regions = mser.detect(gray, None)
print("REGIONS ARE: " + str(regions))

where the dict given to cv2.MSER is
{'_delta':7, '_min_area': 2000, '_max_area': 20000, '_max_variation': .25, '_min_diversity': .2, '_max_evolution': 200, '_area_threshold': 1.01, '_min_margin': .003, '_edge_blur_size': 5}

Python output:

REGIONS ARE: [array([[197,  58],
   [197,  59],
   [197,  60],
   ..., 
   [143,  75],
   [167,  86],
   [172,  98]], dtype=int32), array([[114,   2],
   [114,   1],
   [114,   0],
   ..., 
   [144,  56],
   [ 84,  55],
   [ 83,  55]], dtype=int32)]

Java:

Mat mat = new Mat(bitmap.getWidth(), bitmap.getHeight(), CvType.CV_16S, new Scalar(4));
Mat gray = new Mat(bitmap.getWidth(), bitmap.getHeight(), CvType.CV_16S, new Scalar(4));
Imgproc.cvtColor(mat, gray, Imgproc.COLOR_RGB2GRAY, 4);

FeatureDetector fd = FeatureDetector.create(FeatureDetector.MSER);
MatOfKeyPoint regions = new MatOfKeyPoint();
fd.detect(gray, regions);
System.out.println("REGIONS ARE: " + regions);

Java output:

REGIONS ARE: Mat [ 10*1*CV_32FC(7), isCont=true, isSubmat=false, nativeObj=0x6702c688, dataAddr=0x59add760 ]

where each row of the Mat looks like
KeyPoint [pt={365.3387451171875, 363.75640869140625}, size=10.680443, angle=-1.0, response=0.0, octave=0, class_id=-1]

EDIT:

A mod on the answers.opencv.org forum provided a bit more information (http://answers.opencv.org/question/63733/why-does-python-implementation-and-java-implementation-of-mser-create-different-output/):

unfortunately, it looks, like the java version is limited to the features2d.FeatureDetector interface, which lets you only access KeyPoints (not the actual Regions)

berak (Jun 10 '15)

@berak: So if I understand correctly from the docs, both the java version and the python/C++ version have the features2d.FeatureDetector interface, but the python/C++ version has the additional MSER class to find regions, not just key points? In that case, what do people do? Is it possible to add the C++ MSER class to the OpenCV manager, edit something like the javaFeatureDetector here here, and create a java wrapper for it? Thanks for any advice.

sloreti (Jun 11 '15)

so yes, you can get the rectangles in c++ or python, but not from java. that's a flaw in the design. the javaFeatureDetector is still in use, but to get the rectangles, you'd have to write your own jni interface, i guess. (and distribute your own .so along with your apk)

berak (Jun 12 '15)

Answer 1

You are using two different interfaces to the MSER implementation.

Python cv2.MSER gives you a wrapped cv::MSER, which exposes its operator() to Python as detect:

//! the operator that extracts the MSERs from the image or the specific part of it
CV_WRAP_AS(detect) void operator()( const Mat& image, CV_OUT vector<vector<Point> >& msers,
                                    const Mat& mask=Mat() ) const;

This gives you the nice list of contours interface you are looking for.

In contrast Java uses the javaFeatureDetector wrapper which calls FeatureDetector::detect which is backed by MSER::detectImpl and uses the standard FeatureDetector interface: a list of KeyPoints.

If you want to access the operator() in Java (in OpenCV 2.4) you will have to wrap it up in JNI.