Align profile face image with its frontal face image

Question

I have a profile face:

and a frontal face image:

Output: aligned profile face with reference to frontal face.

Idea: I just need to know which 3 common points I can take,which will be visible on both faces and then use affineTransform and display the aligned profile face

      OR any other **simple method** of doing so

development envi.:c++ and opencv 2.4.2

what I tried:

1. haarcascade feature detection(common detection point in both images=eye) ; it wont detect ear in frontal face
2. OpenCV: Shift/Align face image relative to reference Image (Image Registration) (I get error message)

Answer 1

As discussed here by @bytefish, finding the accurate position of the eyes in a given image is far from trivial. The Haar-cascades for finding the eyes in OpenCV produce too many false positive to be useful, moreover this approach won't be robust to image rotation.

You'll need a robust head pose estimation for aligning face images. Here are two most robust ones (with code available):

• Gary B. Huang, Vidit Jain, and Erik Learned-Miller. Unsupervised joint alignment of complex images. International Conference on Computer Vision (ICCV), 2007. (Project page), (PDF Online available), (Source code)

• X. Zhu, D. Ramanan. Face Detection, Pose Estimation and Landmark Localization in the Wild Computer Vision and Pattern Recognition (CVPR) Providence, Rhode Island, June 2012. (Project page), (PDF Online available), (Source code)

---

For example, using the method described in the second paper, you will get more robust features like that are shown in the following images. And these robust features will, in turn, ensure to generate more robust face alignment performance.