Natural feature tracking with openCV- evaluating the options

Question

In brief, what are the available options for implementing the Tracking of a particular Image(A photo/graphic/logo) in webcam feed using OpenCv?In particular i am trying to collate opinion about the following:

1. Would HaarTraining be overkill(considering that it is not 3d objects but simply Images to be tracked) or is it the only way out?

2. Have tried Template Matching, Color-based detection but these don't offer reliable tracking under varying illumination/Scale/Orientation at all.

3. Would SIFT,SURF feature matching work as reliably in video as with static image comparison?

Am a relative beginner to OpenCV , as is evident by my previous queries on SO (very helpful replies). Any cues or links to what could be good resources for beginning NFT implementation with OpenCV?

Answer 1

Can you talk a bit more about your requirements? Namely, what type of appearance variations do you expect/how much control you have over the environment. What type of constraints do you have in terms of speed/power/resource footprint?

Without those, I can only give some general assessment to the 3 paths you are talking about.

1. Haar would work well and fast, particularly for instance recognition.

Note that Haar doesn't work all that well for 3D unless you train with a full spectrum of templates to cover various perspectives. The poster child application of Haar cascades is Viola Jones' face detection system which is largely geared towards frontal faces (can certainly be trained for many other things)

For a tutorial on doing Haar training using OpenCV, see here.

2. Try NCC or better yet, Lucas Kanade tracking (cvCalcOpticalFlowPyrLK which is a pyramidal as in coarse-to-fine LK - a 4 level pyramid usually works well) for a template. Usually good upto 10% scale or 10 degrees rotation without template changes. Beyond that, you can have automatically evolving templates which can drift over time.

For a quick Optical Flow/tracking tutorial, see this.

3. SIFT/SURF would indeed work very well. I'd suggest some additional geometric verification step to remove spurious matches.

I'd be a bit concerned about the amount of computational time involved. If there isn't significant illumination/scale/in-plane rotation, then SIFT is probably overkill. If you truly need it, check out Changchang Wu's excellent SIFTGPU implmentation. Note: 3rd party, not OpenCV.