Unity UI - Interacting with worldspace UI when cursor is locked

Question

I'm trying to interact with world space UI using a first person controller when Cursor.lockState is set to CursorLockMode.Locked.

world space UI and a character

But when cursor is locked, cursor position is set to (-1, -1), which is told from the Inspector.

cursor position of (-1, -1)

I performed a graphic raycast with EventSystem.RaycastAll, Sreen.width/2 and PointerEventData. EventSystem.current.RaycastAll collects all UI objects in the middle of screen, but no events is sent to them.

I also tried ExecuteEvents.Execute<IEventSystemHandler> to manully send event to UI targets. This works for button when I send 'submit' event to it. Obviously this is not an elegant solution. I have no idea how to send message to slider either.

// manully send a 'submit' event to UI elements
List<RaycastResult> results = new List<RaycastResult>();
void Update() {
    if (Input.GetButtonUp("Fire1")) {
        PointerEventData data = new PointerEventData(EventSystem.current);
        data.position = new Vector2(Screen.width / 2, Screen.height / 2);
        EventSystem.current.RaycastAll(data, results);
        foreach (var result in results) {
            ExecuteEvents.ExecuteHierarchy<ISubmitHandler>(
                result.gameObject, data,
                ExecuteEvents.submitHandler
            );
        }
    }
}

This crazy attempt works when played full-screen on Windows. 2333

[System.Runtime.InteropServices.DllImport("user32.dll")]
public static extern int SetCursorPos ( int x , int y );
void SetCursorPositionToCenter()
{
　　SetCursorPos(Screen.width/2, Screen.height/2);
}

Relavant Resources

Answer 1

I had a lot of problems working with Graphics Raycaster and Global UI elements as well. I tried a lot of things as well but the event system didn't seem to work. But I was simulating a mouse with a touchpad and maybe I didn't completely simulate how a mouse works with UI elements.

I ended up abandoning graphics raycaster and instead add a box collider to all UI elements. That way a regular physics raycaster would be able to detect it and run some code. For the slider, it is just increasing the slider's value as your x or y of your pointer increase.

Hope this helps a little.

Answer 2

What you're doing with the submit handler is similar to the many different ways this can be done. I've not dealt with sliders specifically, because it's not very easy to control sliders in this method (player controller or, in the case of VR, the user's head, does not do straight lines very well). So for slider input we usually go with +/- buttons. Another idea could be that when the user is pointing at a UI element that you release the cursorLock when they click. When click is released you can resume cursor lock.

UI elements are also often in need of states like hover/onEnter, onExit, sometimes pointer up, and usually pointer down. So we've often used Selectable class for this. I think all the interactable UI elements inherit from this so it's a good spot to access the UI element from. This will usually trigger the proper events but it can be a little tricky. Examine some of the many readily available gaze input systems as they're doing the same thing. I don't think I've ever seen one handle sliders though (they're usually meant for VR and gaze-to-slider control would be bad UX).

Cheers and good luck!