Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

OpenCV - Image Stitching

I am using following code to stitch to input images. For an unknown reason the output result is crap! It seems that the homography matrix is wrong (or is affected wrongly) because the transformed image is like an "exploited star"! I have commented the part that I guess is the source of the problem but I cannot realize it. Any help or point is appriciated!

Have a nice day, Ali

void Stitch2Image(IplImage *mImage1, IplImage *mImage2) 
{ 

    // Convert input images to gray 
    IplImage* gray1 = cvCreateImage(cvSize(mImage1->width, mImage1->height), 8, 1); 

    cvCvtColor(mImage1, gray1, CV_BGR2GRAY); 
    IplImage* gray2 = cvCreateImage(cvSize(mImage2->width, mImage2->height), 8, 1); 

    cvCvtColor(mImage2, gray2, CV_BGR2GRAY); 
    // Convert gray images to Mat 
    Mat img1(gray1); 
    Mat img2(gray2); 
    // Detect FAST keypoints and BRIEF features in the first image 
    FastFeatureDetector detector(50); 
    BriefDescriptorExtractor descriptorExtractor; 
    BruteForceMatcher<L1<uchar> > descriptorMatcher; 
    vector<KeyPoint> keypoints1; 
    detector.detect( img1, keypoints1 ); 
    Mat descriptors1; 
    descriptorExtractor.compute( img1, keypoints1, descriptors1 );

/* Detect FAST keypoints and BRIEF features in the second image*/


    vector<KeyPoint> keypoints2; 
    detector.detect( img1, keypoints2 ); 
    Mat descriptors2; 
    descriptorExtractor.compute( img2, keypoints2, descriptors2 ); 
    vector<DMatch> matches; 
    descriptorMatcher.match(descriptors1, descriptors2, matches); 
    if (matches.size()==0) 
            return; 
    vector<Point2f> points1, points2; 
    for(size_t q = 0; q < matches.size(); q++) 
    { 
            points1.push_back(keypoints1[matches[q].queryIdx].pt); 
            points2.push_back(keypoints2[matches[q].trainIdx].pt); 
    } 
    // Create the result image 
    result = cvCreateImage(cvSize(mImage2->width * 2, mImage2->height), 8, 3); 
    cvZero(result); 

   // Copy the second image in the result image 

    cvSetImageROI(result, cvRect(mImage2->width, 0, mImage2->width, mImage2->height)); 
    cvCopy(mImage2, result); 
    cvResetImageROI(result); 

  // Create warp image 
    IplImage* warpImage = cvCloneImage(result); 
    cvZero(warpImage); 

  /************************** Is there anything wrong here!? *******************/ 
   // Find homography matrix 
    Mat H = findHomography(Mat(points1), Mat(points2), 8, 3.0); 
    CvMat HH = H; // Is this line converted correctly? 
   // Transform warp image 
    cvWarpPerspective(mImage1, warpImage, &HH); 
  // Blend 
    blend(result, warpImage);
  /*******************************************************************************/ 

    cvReleaseImage(&gray1); 
    cvReleaseImage(&gray2); 
    cvReleaseImage(&warpImage); 
}
like image 406
Edi Avatar asked May 16 '11 13:05

Edi


2 Answers

This is what I would suggest you to try, in this order:

1) Use CV_RANSAC option for homography. Refer http://opencv.willowgarage.com/documentation/cpp/calib3d_camera_calibration_and_3d_reconstruction.html

2) Try other descriptors, particularly SIFT or SURF which ship with OpenCV. For some images FAST or BRIEF descriptors are not discriminating enough. EDIT (Aug '12): The ORB descriptors, which are based on BRIEF, are quite good and fast!

3) Try to look at the Homography matrix (step through in debug mode or print it) and see if it is consistent.

4) If above does not give you a clue, try to look at the matches that are formed. Is it matching one point in one image with a number of points in the other image? If so the problem again should be with the descriptors or the detector.

My hunch is that it is the descriptors (so 1) or 2) should fix it).

like image 75
KMS Avatar answered Oct 15 '22 13:10

KMS


Also switch to Hamming distance instead of L1 distance in BruteForceMatcher. BRIEF descriptors are supposed to be compared using Hamming distance.

like image 35
Rahul Garg Avatar answered Oct 15 '22 13:10

Rahul Garg