How to handle MotionEvents (Gamepad Joystick Movement, etc) from AccessibilityService

Question

I have an AccessibilityService that takes in input from game controllers (ps5 controller, xbox controller, etc.).

I use the onKeyEvent() method to handle button presses and releases, so I can handle those easily. The problem I am facing is how to handle the Joystick movements and top trigger presses, as I am unaware how to handle them through an AccessibilityService.

Normally, I would simply use onGenericMotionEvent() to handle these MotionEvents, but unfortunately AccessibilityService does not provide me with such a method. I have looked at the docs and official codelabs for almost 3 weeks with no luck, if someone could tell me how to handle MotionEvents through an AccessibilityService I would be very relieved.

The MotionEvents I want to handle are these:

AXIS_X, AXIS_Y, AXIS_Y, AXIS_RZ, AXIS_RY, AXIS_RX, AXIS_HAT_X, AXIS_HAT_Y, AXIS_LTRIGGER, AXIS_RTRIGGER, AXIS_BRAKE,AXIS_GAS.

There may be others depending on the controller, but these are the main ones I need to handle input from my controller.

Regards, 0xB01b

Answer 1

The API doesn't support these because AccessibilityServices can only filter KeyEvents, and the buttons as you say do not produce KeyEvents.

Can you explain what you're building? Understanding the impact of adding this api to the lives of people with disabilities would help us prioritize this work along with other stuff we're planning.

Answer 2

To create an InputMethodService, you can refer to this official document.

First, add service in the Manifest.xml:

<service android:name=".IMService"
        android:permission="android.permission.BIND_INPUT_METHOD">
        <intent-filter>
            <action android:name="android.view.InputMethod"/>
        </intent-filter>
        <meta-data
            android:name="android.view.im"
            android:resource="@xml/method"/>
    </service>

Next, method.xml is need to create under res/xml/ folder:

    <?xml version="1.0" encoding="utf-8"?>
<input-method xmlns:android="http://schemas.android.com/apk/res/android"
    android:isDefault="false"
    android:settingsActivity=".MainActivity">
    <subtype
        android:icon="@mipmap/ic_launcher"
        android:imeSubtypeLocale="en_US"
        android:imeSubtypeMode="keyboard"
        android:label="@string/app_name"/>
</input-method>

Layout of the keyboard is not needed in this case.

Then, create the class IMService implementing the InputMethodService.

The following code is written in Kotlin (for example):

class IMService: InputMethodService() {
companion object {
    private const val TAG = "IMService"
}

override fun onKeyDown(keyCode: Int, event: KeyEvent): Boolean {
    if(event.action == KeyEvent.ACTION_DOWN) {
        if (event.source and InputDevice.SOURCE_GAMEPAD == InputDevice.SOURCE_GAMEPAD) {
            // process key down event here
            return true   // return true if you want the key event to be filtered
        }
    }
    return super.onKeyDown(keyCode, event)
}

override fun onKeyUp(keyCode: Int, event: KeyEvent): Boolean {
    if(event.action == KeyEvent.ACTION_UP) {
        if (event.source and InputDevice.SOURCE_GAMEPAD == InputDevice.SOURCE_GAMEPAD) {
            // process key up event here
            return true
        }
    }
    return super.onKeyUp(keyCode, event)
}

override fun onGenericMotionEvent(event: MotionEvent): Boolean {
    if(event.action == MotionEvent.ACTION_MOVE) {
        if (event.source and InputDevice.SOURCE_JOYSTICK == InputDevice.SOURCE_JOYSTICK) {
            // process motion event here
            return true
        }
    }
    return super.onGenericMotionEvent(event)
}
}

The service lifecycle starts after you enabled and changed the default input method in Settings/../Language and keyboard/Keyboard list and default.