How to Detect Edge of lips using Opencv in iOS?

Question

Im new at OpenCV so please help me out. I want to detect lips edge using OpenCV. So can you give me some links and solution ? Ive checked out normal procedure of detecting face and mouth using OpenCV but accuracy is not there. Ive used "haarcascade_mcs_mouth" to detect mouth in a picture but result was not so good. And I heard about AAM method , but could not get any documents regarding it. please help me...

Answer 1

Lip recognition is a problem in Computer Vision that it is not completely solved. The haar-like classifiers that you've been using (included in OpenCV) perform well in face recognition but for lip recognition better techniques have been developed. You will have to build different algorithms and choose the better one for your purpose. The fact that you are developing for iOS makes the task harder because of additional constraints (memory footprint, CPU, etc.). I have condensed for you a brief overview of the state of the art in lip recognition so you can research further:

Methods for recognising lips can be classified in three big categories:

• Image based techniques: These are based on the hypotesis that skin and lips have different color. The paper [2] is an example of this kind of approach applied for sign language recognition. Color clustering has also been explored by [3]. It assumes that there are two pixel classes in the image: skin and lips. This method is not appropriate if the person has a beard, or is showing his/her teeth, for example.

• Model based techniques: These methods are more robust than the previous ones because they use prior information about the lip shape. However, they are more expensive computationally so they may not be suitable for an implementation on mobile devices. AAM (Active Appearance Models) belong to this group and learn the shape of lips from manually annotated data. In the "External links" section of the Wikipedia article you can see some open source implementations and libraries that can be ported to C++/OpenCV.

• Hybrid techniques: These methods are a combination of image based methods and model based methods. Typically, a color based technique is first applied to the image in order to estimate the lip region position and size; then, a model based technique (like AAM) is applied to the region of interest to extract lip contours. [4] is an example of this technique.

[2] U. Canzler and T. Dziurzyk, "Extraction of Non Manual Features for Videobased Sign Language Recognition". ;In Proceedings of MVA. 2002, 318-321

[3] Leung, Shu-Hung, Shi-Lin Wang, and Wing-Hong Lau. "Lip image segmentation using fuzzy clustering incorporating an elliptic shape function." Image Processing, IEEE Transactions on 13.1 (2004): 51-62.

[4] Bouvier, Christian, P-Y. Coulon, and Xavier Maldague. "Unsupervised lips segmentation based on ROI optimisation and parametric model." Image Processing, 2007. ICIP 2007. IEEE International Conference on. Vol. 4. IEEE, 2007.