How can I recognise a Microsoft Band gesture

Question

From a phone, I can send a message and/or dialog to the paired Microsoft Band, but I would like to get an acknowledgement back from the band to the phone. As there doesn't seem to be any other way, I'm thinking that I am going to have to pick up a gesture such as the user rotating their wrist back and forth once, or possibly shaking their wrist. This sounds like a job for the accelerometer and gyroscope. Collecting information from those should be easy enough, but in terms of recognising a gesture such as a wrist rotation in one direction and then back in the other, can anybody advise please on how to identify this as an intentional gesture, rather than just normal "non-gesture" movements?

Many thanks for any help,

John H.

Answer 1

Start by saving the data and plotting it out as a different sorts of graphs to see if you see a pattern for the gesture. I had a similar problem, I wanted to detect when a user took a bite,- and filter out other hand gestures such as cutting, drinking, and so on.

I recorded 1 min of data and took exactly a predefined number of bites, and otherwise interacted normally. I then smoothed out the data a little bit, and looked at the graph to see if I could see a pattern to see which thresholds I could work with.

I determined some of the factors were: Nr of readings in a set that defines a bite (must be over 20 at 16ms interval) Peak and valley on Y axis Breaks between

I used that to create a computation that finds the 'ultimate' thresholds based on training data provided by the user,- but since a hand gesture such as rotation is more or less the same from person to person you can predefine the thresholds and just let the user set sensitivity for example.

Here are some graphs:

Should say Y data :)

This image might be too blurry, but it all started with some ideas and some whiteboarding where everything from using slope, parabola calculations, acceleration over time and what not was considered. Brute force worked the best, or I just suck at math.

Summary

• Write down what defines your gesture and factors involved
• Record data from accelerometer and gyro and plot some graphs
• Look for common patterns
• Put together an algorithm that would yield correct response based on input data and factors.
• Tweak and refactor.

Shake should be rather easy, just decide which way (up and down or sideways, or both allowed). Set thresholds for the axises, and decide on how many crossing there has to be within what amount of time.

pseudo code on the phone, pardon ugliness :) Something along the lines of...?:

if (totaltime < maxTime)

if(reading.X > something) if(reading.timestamp - itemsForCount.Last().TimeStamp <= maxTimeBetweenShakes) itemsForCount.Add(new item({....})

if (itemsFourCount.Count >= desiredShakes) itemsFourCount.Clear(); if(totaltime > maxTime) itemsFourCount.Clear();

Inspiration: I've done some work with Leap motion and I remember taking a look at their js code to see how the few gestures they had were defined. Here is Js leap motion. C# libs need to be decompiled and that would be against lic. agreement.

Hammer.js is otherwise an interesting OSS where you can see how touch gestures are defined,- lookup swipe and rotate.

There are also some kinect gesture apis worth looking at, but as with leap its different since you usually have a skeletal or point model to work with and a view box for coordinates.