Upper Body Skeleton Detection

Question

For an educational project we are currently working on a basic motion and gesture detection system. Our main goal is to detect a human body on a camera stream. Using this information we want to detect a basic skeleton of the body to do a further detection of the gestures the person does.

We are using EmguCV / OpenCV to process our stream. I found this video on Youtube: http://www.youtube.com/watch?v=fYZtmkfWh5g. He is able to detect the upper body and place a basic skeleton in it, but how? He does not mention any algorithms used.

We already tried to detect the body contour using a background subtraction (BackgroundSubtractorMOG2) but the camera noise and lightning conditions made us fail.

Does anybody have an idea to detect the body exactly like shown in the video?

Answer 1

I think that you should implement next steps:

1. Detect face.
2. Detect hands.
3. Estimate scale of body image.
4. Place upper body kinematic model (skeleton), using head position information and set its scale usung detected head size.
5. Match models head and hands with detected before positions of human head and hands.
6. Update inverse kinematics of model for get human pose estimation.
7. I suppose, that here should be very good to use some statistical filter, like particle filter or Kalman filter.
8. Go to step 1.

For kinematic model you can use some 2D physics endine (Box2D for example).