Finding Pose variation of a face using Opencv

Question

In an image there is a face, which is tilted a bit and has a slight pose variation. Using opencv i detected the co-ordinates of eyes, nose, and mouth. Using these 4 points is it possible to re-align this face into an ideal frontal face ? I tried out perspective transformation, which did work for a few images, but gave horrible results for most of the pics. Is there any other way of re-aligning the image into an ideal frontal face without any tilt or pose variation?

Answer 1

I think you can refer the Procrustes Analysis to get more help. The idea would be to find a rigid transformation matrix from the ideal human face to the actual human face using a set of known points. Then use the inverse of this transformation matrix to each point on the actual human face to get back the ideal one.

Here you have a set of known points (two eyes, tip of nose and mouth). You can assume an ideal position for them. From the image detection using OpenCV, you get 4 corresponding points. Also you could refer to Kabsch algorithm if you can assume only rotations are possible, which I think might make sense in your case. There is a nice question here explaining the above topics. I hope the above references help you.