How can you implement non-standard Leap Motion gestures?

Question

The Leap Motion API only supports four standard gestures: circle, swipe, key tap and screen tap. In my application I need other gestures, but I do not know how can I add them or if it is even possible to add more gestures. I read the API and it was no help.

In my application I want to allow the user to hold an object and drag it. Is that possible using the Leap Motion API? If so, how can I do this?

Answer 1

You will need to create your own methods to recognize the starting point, the process, and the ending point of a gesture.

Starting point: How will your program recognize you are trying to hold something? A simple gesture I can think of is 2 fingers linked with 1 palm. So in the frame, if you see 2 fingers linked with 1 palm and the fingers are maybe 10-20 mm apart, you can recognize it as a gesture to hold something. When these conditions are fulfilled, the program will recognize the gesture and you can write some code inside these conditions.

For a very ugly example in C#:

Starting point: Boolean gesture_detected = false;

Frame frame = controller.Frame();
HandList hands = controller.Hands;

if (hands.Count == 1)
{
   foreach (Hand hand in hands)
   {
       if (hand.fingers.Count == 2)
       {
           int fingerA_x,fingerB_x;
           foreach (Finger finger in hand.fingers)
           {
               if(fingerA_x == 0)
               {
                   fingerA_x = finger.x;
               } else
               {
                   fingerB_x = finger.x;
               }
           }
       }
   }

   if((fingerA_x - fingerB_x) < 20)
   {
         //Gesture is detected. Do something...
         gesture_detected = true;
   }
}

Process: What is your gesture trying to do? If you want to move around, you will have to call a mouse method to do a drag. Search for the method mouse_event() in C++ under PInvoke using the event MOUSEEVENTF_LEFTDOWN.

Ending point: After you finish dragging, you need to call a mouse method event like MOUSEEVENTF_LEFTUP to simulate a mouse drag that has finished. But how will your program detect when you should stop the drag? Most logical way is if the gesture is no longer being detected in the frame. So write an else condition to process the alternate scenario.

       if (!gesture_detected)
       {
            // Do something
       }