Head Pose Estimation with OpenCV, C++ and Image 2D - Geometric Method - Roll, Yaw and Pitch

Question

I'm trying to find the three angles of the face of a person, based on a 2D image .

I'm using OpenCV with HaarCascade to find the face, eyes, nose and mouth. But I don't found any geometric method that can help me to find the angles X, Y and Z (Roll, Pitch and Yaw).

Could someone help me showing some method in c++ or java that works?

Answer 1

Given a single image and no other information, there is no single solution for the angles. Consider the case of just Yaw. Projected onto the 2d plane, this is visible as a small change in the projected distance between eyes and the placement of the eyes with respect to the nose/mouth. This distance is not a constant from person to person, however.

One typical way around this is to require that the user 'calibrate' their face by looking directly at the camera for the nominal '0' angles. At this point, you now have reference lengths against which you can compare subsequent images.

The lengths are still not quite enough information, however, as the amount that the apparent projected distances change depends on the optics and the distance of the face from the camera. The optics you usually configure manually; the distance you can estimate by assuming 'average' facial dimensions and assuming the 'nominal' image matches those dimensions perfectly. You can make this adjustable if you find that it's over- or under- estimating the rotations for a particular face.

Once you have all these assumptions in place, it's fairly simple geometry. You can estimate roll from the line from the eyes through the nose to the mouth. You can measure the spacing between the eyes to estimate yaw. Finally, you can estimate pitch using the spacing between eyes/mouth or eyes/nose. Bear in mind, these assumptions work best when the face is still fairly close to nominal.