How to detect Facial Sideview Left ear, Sideview Nose, Sideview mouth in iOS Application using OpenCV?

Question

I need help with face profiling through image in an iOS application.

I am trying to detect left ear, nose and mouth in a given image. So far I tried OpenCV, I found voila's haar classifiers but this haar classifier does not detect left ear.

I need to perform this detection without going to server/online.

Is OpenCV good choice for this? Any sample codes you can share to achieve this functionality would be great.

What can be other choices to achieve this functionality?

Answer 1

I think only using part templates (e.g., viola's haar-classifiers) will not work in your case. The parts you want to detect are very small and will be fully/partially occluded most of the time. My suggestion would be to use graphical models based methods, i.e., active appearance models, pictorial structures, etc. This will not only allow you to exploit spatial constraints (i.e, mouth should be always below nose, etc.), but also works when one or few of the parts are occluded. Probably you can start with following publicly available codes:

1. http://cmp.felk.cvut.cz/~uricamic/flandmark/index.php#structclass
2. http://www.iai.uni-bonn.de/~gall/projects/facialfeatures/facialfeatures.html

Both codes are in C++, and will allow you to detect facial body parts, but I think ears are not included in both. May be you can try adding additional parts by slightly modifying the source code, and also training your own part-templates for the missing parts.

PS. I am not an iOS developer, so I am not sure if iOS can afford such models, but on normal computers they are sufficiently real time for normal size images.