Feature Extraction Methods for Hand gesture/posture recognition

Question

I am currently working on a Sign Language Recognition application, where I would like to use a Hidden Markov Model as the classification stage, meaning that I will classify a gesture/posture to obtain the relevant letter or word.

I have currently completed the first stage where I am detecting the hand. Currently I can obtain a number of parameters (features) which I can use for my machine learning stage such as:

• convex hull of hand
• convexity defects
• centroid of the hand
• bounding rotated ellipses/rectangles (e.g. obtain any angle needed in terms of rotation)
• contour of the hand
• moments (I am not sure what these are extactly)

These are all possible to do through openCv.

My question: once I have all these features, how can I execute the 'Feature Extraction' stage? i.e. if a machine learning algorithm, in this case the HMM requires a set of probabilities, how can I use the above information?

One idea I have is to create a special data structure with such information which uniquely identifies each gesture, but how do I feed it to the machine learning technique? (in this case the Hidden Markov Model)

Can any one be able to guide me as to what I should at least search for at this particular stage or guide me to show what is actually the real difficulty I have?

Answer 1

Once you have your set of observations ready, you could feed it to the Viterbi Algorithm to detect the best state sequence that may have produced these observations. Also, you can train your HMM over a data set of samples using the Baum-Welch algorithm. You could have a look at my blog post which is a simple explanation of recognizing dynamic hand gestures using HMM (although I am NOT using openCV or scanning the contour of the hand). Hope this can help you in getting a general idea about the processing and learning phase.