Kinect SDK 1.7 | Change KinectCursor image and size

Question

I have downloaded Kinect SDK 1.7, toolkit and have played with following samples.

• ControlBasics WPF
• InteractionGallery WPF.

I figured out that Kinect Toolkit internally uses the interaction frame to detect the hand position/gesture and accordingly maps it with the Kinect Controls.

I have a requirement where I want to capture a grip event on a Kinect Tile Button. Since the default KinectTileButton does not provide a Grip event. I added a grip event handler on my button.

KinectRegion.AddHandPointerGripHandler(kinectButton, OnHandPointerCaptured);

private void OnHandPointerCaptured(object sender, HandPointerEventArgs handPointerEventArgs) 
{ 
    // Add code here
}

I placed a debug breakpoint inside the OnHandPointerCaptured method and was able to receive proper hits when I grip on the KinectTileButton. But for some reason I do not see the KinectCursor image change to a grip as it happens on the KinectScrollViewer control. I tried setting up the isGripTarget property in the KinectButtonBase class but it doesn't help.

private void InitializeKinectButtonBase() 
{ 
    KinectRegion.AddHandPointerPressHandler(this, this.OnHandPointerPress); 
    KinectRegion.AddHandPointerGotCaptureHandler(this, this.OnHandPointerCaptured); 
    KinectRegion.AddHandPointerPressReleaseHandler(this, this.OnHandPointerPressRelease); 
    KinectRegion.AddHandPointerLostCaptureHandler(this, this.OnHandPointerLostCapture); 
    KinectRegion.AddHandPointerEnterHandler(this, this.OnHandPointerEnter); 
    KinectRegion.AddHandPointerLeaveHandler(this, this.OnHandPointerLeave); 

    // Use the same OnHandPointerPress handler for the grip event 
    KinectRegion.AddHandPointerGripHandler(this, this.OnHandPointerPress); 

    //Set Kinect button as Grip target
    // KinectRegion.SetIsPressTarget(this, true); 
    KinectRegion.SetIsGripTarget(this, true);                
}

How do I change the KinectCursor image from openhand icon to grip, and also the size.

Answer 1

Here's what i found-

1) Cursor Image and Size - Both these samples use Microsoft.Kinect.Toolkit.Controls project that defines the Custom Controls (KinectTileButton,KinectScrollViwer,KinectRegion etc.) used in these samples. The Kinect Cursor image and size is defined as a resource in Microsoft.Kinect.Toolkit.Controls ->Themes->Generic.xaml. Search for following in the file -

  <Style TargetType="{x:Type local:KinectCursor}">

You can modify this as per your need. Wasn't able to find any properties/hooks exposed to control this directly from the UI.

2) Grip event on Tile Button - Kinect Button Supports various events which can be subsribed to and acted upon on your desire. see this Hand over button event in Kinect SDK 1.7

3) Change Cursor image to Grip on Tile Button - Microsoft.Kinect.Toolkit.Controls project uses KinectCursorVisualizer.cs and KinectCursor.cs to render the virtual hand cursor on the UI. The Open Hand/Grip visual state is controlled via this Dependency Property defined in KinectCursor.cs.

public static readonly DependencyProperty IsOpenProperty = DependencyProperty.Register(
        "IsOpen",
        typeof(bool),
        typeof(KinectCursor),
        new UIPropertyMetadata(true, (o, args) => ((KinectCursor)o).EnsureVisualState()));

A quick find all references on the property IsOpen tells that the only place where this property is set is in KinectCursorVisualizer.cs-> OnHandPointersUpdated method. Line- 229

// Set open state
 cursor.IsOpen = !pointer.IsInGripInteraction;

And this pointer.IsInGripInteraction property is set at KinectAdapter.cs Line 678

  handPointer.IsInGripInteraction = newIsInGripInteraction;

If you look at the code just above this line you'll find that this property is only set to true if the target element has a QueryInteractionStatusHandler defined and it sets args.Handled , args.IsInGripInteraction property to true.

Since KinectScrollViewer has this handler defined you see a grip image.

 private void InitializeKinectScrollViewer()
        {
            KinectRegion.AddHandPointerGotCaptureHandler(this, this.OnHandPointerCaptured);
            KinectRegion.AddHandPointerLostCaptureHandler(this, this.OnHandPointerLostCapture);
            KinectRegion.AddHandPointerEnterHandler(this, this.OnHandPointerEnter);
            KinectRegion.AddHandPointerMoveHandler(this, this.OnHandPointerMove);
            KinectRegion.AddHandPointerPressHandler(this, this.OnHandPointerPress);
            KinectRegion.AddHandPointerGripHandler(this, this.OnHandPointerGrip);
            KinectRegion.AddHandPointerGripReleaseHandler(this, this.OnHandPointerGripRelease);
//This is the QueryInteractionStatusHandler
            KinectRegion.AddQueryInteractionStatusHandler(this, this.OnQueryInteractionStatus);
            KinectRegion.SetIsGripTarget(this, true);
            this.scrollMoveTimer.Tick += this.OnScrollMoveTimerTick;
            this.scrollViewerInertiaScroller.SlowEnoughForSelectionChanged += this.OnSlowEnoughForSelectionChanged;

            // Create KinectRegion binding
            this.kinectRegionBinder = new KinectRegionBinder(this);
            this.kinectRegionBinder.OnKinectRegionChanged += this.OnKinectRegionChanged;    
        }

but KinectTileButton (extends KinectButtonBase) does not have this handler defined

 private void InitializeKinectButtonBase()
        {
            KinectRegion.AddHandPointerPressHandler(this, this.OnHandPointerPress);
            KinectRegion.AddHandPointerGotCaptureHandler(this, this.OnHandPointerCaptured);
            KinectRegion.AddHandPointerPressReleaseHandler(this, this.OnHandPointerPressRelease);
            KinectRegion.AddHandPointerLostCaptureHandler(this, this.OnHandPointerLostCapture);
            KinectRegion.AddHandPointerEnterHandler(this, this.OnHandPointerEnter);
            KinectRegion.AddHandPointerLeaveHandler(this, this.OnHandPointerLeave);

            KinectRegion.SetIsPressTarget(this, true);

        }

How to define this handler ? - Simple add following to your UI. Can be added to constructor

//Add the handler
 KinectRegion.AddQueryInteractionStatusHandler(kinectButton, OnQuery);

Define the Handler

 //Variable to track GripInterationStatus
 bool isGripinInteraction = false;

        private void OnQuery(object sender, QueryInteractionStatusEventArgs handPointerEventArgs)
        {

            //If a grip detected change the cursor image to grip
            if (handPointerEventArgs.HandPointer.HandEventType == HandEventType.Grip)
            {
                isGripinInteraction = true;
                handPointerEventArgs.IsInGripInteraction = true;
            }

           //If Grip Release detected change the cursor image to open
            else if (handPointerEventArgs.HandPointer.HandEventType == HandEventType.GripRelease)
            {
                isGripinInteraction = false;
                handPointerEventArgs.IsInGripInteraction = false;
            }

            //If no change in state do not change the cursor
            else if (handPointerEventArgs.HandPointer.HandEventType == HandEventType.None)
            {
                handPointerEventArgs.IsInGripInteraction = isGripinInteraction;
            }

            handPointerEventArgs.Handled = true;
        }

You may have to tweak this as per your requirement. Happy Kinecting :)