How to handle mouse events in iOS 13 objective-c

Question

iOS 13 finally introduced support for the mouse. I have been searching in vain for documentation on how to handle mouse events. In macOS 10+ there is NSEvent class https://developer.apple.com/documentation/appkit/nsevent as part of AppKit. What is the corresponding class in iOS 13? Can someone point me to some documentation/examples, please? I have tried the code in https://developer.apple.com/library/archive/documentation/Cocoa/Conceptual/EventOverview/HandlingMouseEvents/HandlingMouseEvents.html but it does not work in iOS 13. I am using Xcode 11. I am particularly interested in getting the mouse event information (e.g., pressedMouseButtons, mouseLocation, buttonNumber).

- (void)mouseDown:(NSEvent *)theEvent {
    [self setFrameColor:[NSColor redColor]];
    [self setNeedsDisplay:YES];
}
- (void)mouseUp:(NSEvent *)theEvent {
    [self setFrameColor:[NSColor greenColor]];
    [self setNeedsDisplay:YES];
}

Answer 1

There seem to be no public APIs for this feature.

For some of what you want here, there is starting in iPadOS 13.4!

All "mouse" events on iOS 13 register as regular touch events, just like events produced by AssistiveTouch. You're unable to check for the source of these events.

This is correct. Lets dig in...

Background and links

On iPad, starting with iPadOS 13.4, trackpads and mice are supported without any accessibility features enabled. Note that iPhone does not get this native support. The API I discuss below still applies, but will only work on the iPhone with pointing device support for AssistiveTouch enabled.

We go over the event and gesture API here: https://developer.apple.com/videos/play/wwdc2020/10094/

If you check out that video in the Developer app, there are code snippets from the video included.

And some of the compatibility mode affordances I describe below are detailed here: https://developer.apple.com/documentation/bundleresources/information_property_list/uiapplicationsupportsindirectinputevents

(edit) GameController.framework does show all the connected mice as I mention below, and you can set up change handlers for each. However, there's no way to see the ID from these devices there:

https://developer.apple.com/documentation/gamecontroller/gcmouse/3626182-mice https://developer.apple.com/videos/play/wwdc2020/10617/

Pointer movement and clicks

Pointer movement is driven by UIEvent.EventType.hover. You handle that with a UIHoverGestureRecognizer.

Clicks on a pointing device are transformed into a UITouch instances. As a compatibility affordance, these are all UITouch.TouchType.direct by default. That's the same as a finger-based touch.

If you opt into UIApplicationSupportsIndirectInputEvents in your application's info.plist key, clicks on pointing devices are transformed into UITouch.TouchType.indirectPointer. That will allow you to distinguish these touches from finger-based touches, and you can target these touches specifically using API like UIGestureRecognizer.allowedTouchTypes.

Button mask

With UIApplicationSupportsIndirectInputEvents, you'll be able to recognize that a touch from a pointing device happened. To recognize the specific button, you'll want to look at either UIEvent.buttonMask or UIGestureRecognizer.buttonMask. Note that UIGestureRecognizer's property is from the last event processed.

.buttonMask will show you the set of buttons pressed during an event.

Conclusion

As I understand it you wanted to accomplish the following:

1. Recognize clicks from a pointing device
2. Determine which button was pressed
3. Determine which device the click came from.

For #1, you can use UIApplicationSupportsIndirectInputEvents to get the new UITouch.TouchType.indirectPointer, which will allow you to distinguish these touches.

For #2, you can use .buttonMask on either UIEvent or UIGestureRecognizer.

For #3, there is not currently API to achieve this on iOS.

(edit) A co-worker mentioned that GameController.framework might help with some of this.

https://developer.apple.com/documentation/gamecontroller

You'll be able to see all the connected mice with GCMouse.mice() and be notified when they connect:

https://developer.apple.com/documentation/gamecontroller/gcmouse/3626182-mice

Unfortunately, you won't be able to see the ID of the mouse here either. Still, it gives you something to consider.

Check out this video that dives into more of the details on GameController.framework support for pointing devices and keyboards:

https://developer.apple.com/videos/play/wwdc2020/10617/