How to track head position

Question

I want to do something similar to what Johhny Lee did in his Wii head tracking http://www.youtube.com/watch?v=Jd3-eiid-Uw&feature=player_embedded

But I want to use the Kinect. Since Microsoft's sdk exposes the skeletal joints, I had hoped I might be able to just use that to get the head position. The problem is that I want to do this with my desktop computer and its monitor. If I put the Kinect sensor right next to my monitor and sit at the desk. pretty much just my head and neck are visible to the sensor, so the skeletal tracking doesnt pickup on my head position.

Is anyone familiar with a head tracking project using the Kinect? Preferably in C#

Answer 1

I think for this application you are not able to use skeleton tracking provided by any framework such as Microsoft's SDK or OpenNI.

I would propose to segment the head of the user by applying a depth threshold to the raw depth data. This should result in background reduction. I think there are already existing methods which do this.

As second step you woud like to have something like an axis inside the segmented user. The simplest approach would be to use opencv fitEllipse. The major axis of the returned ellipse combined with the depth information gives you this axis.

This approach only works when most segmented points belong to the head of the user. If you are further away you have to think about a method to segment only the head. The ellipse fitting should always work.

Answer 2

There are limitations on the offical Kinect for Windows SDK, which fall in-lines with the guidance provided for the XBox and XDK in that you need to be 1.2m to 3.5m away from the sensor to be able to use the Kinect sensor. This limitation is actually reduced in alternate SDKs such as the OpenNI/NITE libraries that would allow you to detect skeletons/objects closer to the sensor.

The problem you will also have with the skeltal input is it will only detect the position of the head in proportion to the skeleton, but not if you rotate your head side to side. To achieve this you would not to use the raw depth streams and some smarts around object recognition which is somewhat more complicated.

In the past I've used this commercial .NET API which uses a webcam to track head movements, and achieves what you are after: http://luxand.com/facesdk/index2.php