Android OpenCV: drawing matches with feature2d

Question

I can't get this function to work. Basically I am doing feature detection and matching with a reference image. I want to overlay the matched features on top of my input image. Here is my code:

public Mat startProcessing(Mat inputImage) {

    Imgproc.cvtColor(inputImage, rgb, Imgproc.COLOR_GRAY2RGB);
    myFeatures.detect(rgb, keypoints);
    descriptorExtractor.compute(inputImage, keypoints, imageDescriptors);
    descriptorMatcher.match(templateDescriptors, imageDescriptors, matches);

    Features2d.drawMatches(rgb, keypoints, templateImage, templateKeypoints, matches, rgb); 

    Imgproc.cvtColor(rgb, outputImage, Imgproc.COLOR_RGB2RGBA);
    return outputImage;
}

It's Features2d.drawMatches which is specifically the problem. According to the Android error I've gotten, some dimensions aren't lining up. Truth be told, I'm not even sure how drawMatches is supposed to work on Android because there's no documentation anywhere.

What I want to be able to do is draw the matches on top of the inputImage. I've gotten Feature2d.drawKeypoints to work, but I'm unsure how I would go about converting MatOfDMatch matches to a MatOfKeypoint object. Also I'm not even sure if that's how the MatOfDMatch works - doesn't it have some associativity inside of it corresponding to the descriptors?

Sorry for being all over the place, I'm having a really hard time using OpenCV with Android. The C++ documentation out there is good, but there's little in the way of Android materials. Thanks for any help!

Answer 1

First thing that I've noticed is that you are converting images from one color space to another. It's confusing a little bit. If you want to draw matches on greyscale images, it is not necessary to perform this image conversion. Other thing, you are detecting keypoints on RGB image and then extracting descriptors from original image(as I can see it is greyscale)

Also I suggest you to check inside of your function if images are of the same type. Just print in log inputImage object and see if both are of type CV_8UC3 or equivalent. Basically your code should be like this:

Mat imageOut = inputImage.clone();
Features2d.drawMatches(inputImage, keypoints, templateImage, templateKeypoints, matches, imageOut);
Highgui.imwrite("result_match.jpeg", imageOut); 

Answering to another your questions:

Q:I'm unsure how I would go about converting MatOfDMatch matches to a MatOfKeypoint object. A:You do not need to do it, because drawMatches function receives MatOfKeypoint of input image and also of template image. Then when you passing MatOfDMatch it draws matches between matched received kypoints.

Q:Also I'm not even sure if that's how the MatOfDMatch works - doesn't it have some associativity inside of it corresponding to the descriptors? A:Yes it have. If you will do matches.toList(0).queryIdx and matches.toList(0).trainIdx you will get the index of inputImage's keypoint, which matches with templateImage's keypoint of the first match.