How to make the sample run Open CV

Question

I am trying to make Open CV project sample for Template Matching as explained here . Steps i did so far includes :

Downloaded and imported Open CV framework in my project changed the .m extension files to .mm and in the .pch file i have included the code

 #ifdef __cplusplus
 #import <opencv2/opencv.hpp>
 #endif

 #ifdef __OBJC__
 #import <UIKit/UIKit.h>
 #import <Foundation/Foundation.h>
 #endif

I have also downloaded and imported the MatchTemplate_Demo.cpp file from the link. But here having library linking issue

 ld: warning: directory not found for option '-L/Users/G1/Desktop/Xcode'
   ld: warning: directory not found for option '-Lprojects/FirstOpenCv/opencv/lib/debug'
   ld: library not found for -lopencv_calib3d
   clang: error: linker command failed with exit code 1 (use -v to see invocation)

I followed the same step to include the library as given here.

2) Add $(SRCROOT)/opencv to header search path and $(SRCROOT)/opencv/lib/debug for library search path for debug configuration and $(SRCROOT)/opencv/lib/release for release build.

3) Add OpenCV libs to linker input by modifying "Other Linker Flags" option with "-lopencv_calib3d -lzlib -lopencv_contrib -lopencv_legacy -lopencv_features2d -lopencv_imgproc -lopencv_video -lopencv_core".

Now can please any one tell me how should i make the project run. I have taken the source and template image and imported in the project.

I have basically ViewController.h and ViewController.mm file now i don't know what should i code in these files to see the result.

Also Step 2 : I need to scan the image in real time using camera view (so that when i place my camera over the source image it should scan and find the template).

On following this link i got Linker error while importing the .cpp file :

ld: 1 duplicate symbol for architecture i386 clang:
error: linker command failed with exit    code 1 (use -v to see invocation) 

Please can any one suggest me how should i implement it.

Answer 1

You have three interrelated questions here:

1/ how to get openCV framework to run in an iOS project
2/ how to get the Template Matching c++ sample code to run in an iOS project
3/ how to do live template matching with the camera view

1/ how to get openCV framework to run in an iOS project

• Download and import the openCV framework as you describe
• Change the .pch file as you describe

• check that c++ standard library is set to libc++ in your target build settings (this is the default for new projects)

• Don't just import demo.cpp without making changes as described below (it is a 'raw' c++ progam with it's own main function, and needs alterations to work as part of an iOS/Cocoa project).

• Don't mess with header search paths, other linker flags etc, this isn't necessary if you have imported the prebuilt framework from openCV.org.

• Don't change your .m files to .mm unless you know that you need to. My advice is to keep your c++ code separate from your objective-C code as far as practicable, so most of you files should be .m files (objective-C) or .cpp files (c++). You only need .mm prefixes for "objective-C++" where you intend to mix objective-C and c++ in the same file.

2/ how to get the Template Matching c++ sample code to run in an iOS project

We are going to set this up so that your iOS viewController - and the bulk of your iOS code - does not need to know that the image is processed using openCV/C++, and likewise the C++ code doesn't need to know where it's input or output image data is being routed to. We do this by making a small wrapper class between the two that translates objective-C method calls to c++ class member functions and back. We will also set up a category on UIImage to translate image formats from iOS-friendly UIImage to openCV-native cv::Mat.

UIImage+OpenCV Category

You need some utility methods to convert from UIImage to cv::Mat and back. A good place to put these is in a UIImage category. In XCode: File>New FIle>Cocoa Touch>Objective-C category will set you up. Call the category OpenCV and make it a category on UIImage. This .m file you will want to change to .mm as it will need to understand c++ types from the openCV framework.

The header should look something like this:

#import <UIKit/UIKit.h>

@interface UIImage (OpenCV)

    //cv::Mat to UIImage
+ (UIImage *)imageWithCVMat:(const cv::Mat&)cvMat;

    //UIImage to cv::Mat
- (cv::Mat)cvMat;

@end

The .mm file should implement these methods by closely following this openCV.org code sample adapted to work as category methods (eg you don't pass a UIImage into the instance method, but refer to it using self).

You can use the category methods as if they are UIImage class and instance methods like this:

UIImage* image = [UIImage imageWithCVMat:matImage];  //class method

cv::Mat matImage = [image cvMat];  //instance method

openCV wrapper class

Make a wrapper class to convert your objective-C method (called from a viewController) to a c++ function

header something like this

//  CVWrapper.h
#import <Foundation/Foundation.h>

@interface CVWrapper : NSObject

+ (NSImage*) templateMatchImage:(UIImage*)image
                          patch:(UIImage*)patch
                         method:(int)method;
@end

We send in the template image, patch image and template matching method, and get back an image showing the match

implementation (.mm file)

//  CVWrapper.mm
#import "CVWrapper.h"
#import "CVTemplateMatch.h"
#import "UIImage+OpenCV.h"

@implementation CVWrapper


+ (UIImage*) templateMatchImage:(UIImage *)image
                          patch:(UIImage *)patch
                         method:(int)method
{
    cv::Mat imageMat = [image cvMat];
    cv::Mat patchMat = [patch cvMat];

    cv::Mat matchImage = 
           CVTemplateMatch::matchImage(imageMat, 
                                        patchMat,
                                        method);

    UIImage* result =  [UIImage imageWithCVMat:matchImage];
    return result;
}

We are effectively taking a standard objective-C method and UIImage types and translating them into a call to a C++ member function with c++(openCV framework) types, and translating the result back to a UIImage.

C++ TemplateMatch class

Header:

//  TemplateMatch.h

#ifndef __CVOpenTemplate__CVTemplateMatch__
#define __CVOpenTemplate__CVTemplateMatch__

class CVTemplateMatch
{
public:
    static cv::Mat matchImage (cv::Mat imageMat,
                               cv::Mat patchMat, 
                               int method);
};

#endif /* defined(__CVOpenTemplate__CVTemplateMatch__) */


@end

Implementation:

This is the Template Match openCV example code, reworked as a class implementation:

//  TemplateMatch.cpp
/*
 Alterations for use in iOS project
 [1] remove GUI code (iOS supplies the GUI)
 [2] change main{} to static member function 
     with appropriate inputs and return value
 [3] change MatchingMethod{} signature 
     to return Mat value
 */

#include "CVTemplateMatch.h"

    //[1] #include "opencv2/highgui/highgui.hpp"
#include "opencv2/imgproc/imgproc.hpp"
#include <iostream>
#include <stdio.h>

using namespace std;
using namespace cv;

    /// Global Variables
Mat img; Mat templ; Mat result;
    //[1] char* image_window = "Source Image";
    //[1] char* result_window = "Result window";

int match_method; 
    //[1]  int max_Trackbar = 5; 

    /// Function Headers
Mat MatchingMethod( int, void* );  //[3] (added return value to function)

    // [2] /** @function main */
    // [2] int main( int argc, char** argv )

Mat CVTemplateMatch::matchImage (Mat image,Mat patch, int method)
    // [2]
{
        /// Load image and template
        //[2]  img = imread( argv[1], 1 );
        //[2] templ = imread( argv[2], 1 );

    img = image;           //[2]
    templ = patch;         //[2]
    match_method = method; //[2]

        /// Create windows
        //[1] namedWindow( image_window, CV_WINDOW_AUTOSIZE );
        //[1] namedWindow( result_window, CV_WINDOW_AUTOSIZE );

        /// Create Trackbar
        //[1] char* trackbar_label = "Method: \n 0: SQDIFF \n 1: SQDIFF NORMED \n 2: TM CCORR \n 3: TM CCORR NORMED \n 4: TM COEFF \n 5: TM COEFF NORMED";
        //[1] createTrackbar( trackbar_label, image_window, &match_method, max_Trackbar, MatchingMethod );

    Mat result = MatchingMethod( 0, 0 );

        //[1] waitKey(0);
        //[2] return 0;
    return result;  //[2]
}


    //[3] void MatchingMethod( int, void* )
Mat MatchingMethod( int, void* )

{
        /// Source image to display
    Mat img_display;
    img.copyTo( img_display );

        /// Create the result matrix
    int result_cols =  img.cols - templ.cols + 1;
    int result_rows = img.rows - templ.rows + 1;

    result.create( result_cols, result_rows, CV_32FC1 );

        /// Do the Matching and Normalize
    matchTemplate( img, templ, result, match_method );
    normalize( result, result, 0, 1, NORM_MINMAX, -1, Mat() );

        /// Localizing the best match with minMaxLoc
    double minVal; double maxVal; Point minLoc; Point maxLoc;
    Point matchLoc;

    minMaxLoc( result, &minVal, &maxVal, &minLoc, &maxLoc, Mat() );

        /// For SQDIFF and SQDIFF_NORMED, the best matches are lower values. For all the other methods, the higher the better
    if( match_method  == CV_TM_SQDIFF || match_method == CV_TM_SQDIFF_NORMED )
        { matchLoc = minLoc; }
    else
        { matchLoc = maxLoc; }

        /// Show me what you got
    rectangle( img_display, matchLoc, Point( matchLoc.x + templ.cols , matchLoc.y + templ.rows ), Scalar::all(0), 2, 8, 0 );
    rectangle( result, matchLoc, Point( matchLoc.x + templ.cols , matchLoc.y + templ.rows ), Scalar::all(0), 2, 8, 0 );

        //[1]  imshow( image_window, img_display );  
        //[1] imshow( result_window, result );       

    return img_display; //[3] add return value
}

Now in your viewController you just need to call this method:

UIImage* matchedImage = 
       [CVWrapper templateMatchImage:self.imageView.image
                               patch:self.patchView.image
                              method:0];

with no c++ in sight.

3/ Template Matching with live camera view

The short answer: matchTemplate is not going to work too well in a live camera context. The algorithm is looking for a match in the image with the same scale and orientation as the patch: it slides the patch tile across the image at it's original orientation and size comparing for best match. This is not going to yield great results if the image is perspective-skewed, a different size or rotated to a different orientation.

You could look instead at OpenCV's Feature Detection algorithms, some of which have been moved to non-free. Here is a nice description of SIFT to give you the idea. For video capture you might also want to look at cap_ios.h in opencv2/highgui: here is a tutorial.