Why is 'air tap' gesture not working on HoloLens1 in my Unity/MRTK app?

Question

I have a Unity app that I'd like to integrate with the Microsoft Mixed Reality Toolkit (MRTK). When I add the MRTK (v2.1 or v2.2) package to my Unity project, I can simulate the "air tap" gesture in the Unity Editor and the app registers the click. However, when I publish the app to my HoloLens1 (or run the holographic emulator within Unity), the "air tap" gesture does not register a click.

Unity: 2018.4.x

MRTK: both v2.1 / v2.2

Unity scene setup:

• has an EventSystem with a Standalone Input Module
• has a character that is the Main Camera
• has imported MRTK
• Mixed Reality Toolkit Profile is set to DefaultHoloLens1ConfigurationProfile

When I run the scene with the above setup, the air tap registers in the Unity Editor (by pressing space bar + click to simulate), but it does not register in the HoloLens1.

Is there some part of the setup I am missing? Perhaps another input component to be added to something in my scene?

Answer 1

To respond to click events from MRTK, you need listen for MRTK pointer events instead of the Unity input events that the MRE is likely listening to. A good event to listen for would be the OnPointerClicked event, which will trigger when a hand is tapped, or when a motion controller is clicked, or if you say the word "select". Normally the input handler will only respond if you are hovering over the object, so to respond to these inputs globally you need to register as a global input handler.

Here is an example of a script that will print some text whenever the pointer is clicked:

using Microsoft.MixedReality.Toolkit;
using Microsoft.MixedReality.Toolkit.Input;
using UnityEngine;

public class DetectTapExample : MonoBehaviour
{

    public void Start()
    {
        PointerHandler pointerHandler = gameObject.AddComponent<PointerHandler>();
        pointerHandler.OnPointerClicked.AddListener((evt) => Debug.Log("Tap Detected " + Time.time));
        // Make this a global input handler, otherwise this object will only receive events when it has input focus
        CoreServices.InputSystem.RegisterHandler<IMixedRealityPointerHandler>(pointerHandler);
    }
}

You can learn more about Pointers here and the MRTK Input System here