ADB Accessibility Focus Change

Question

I would like to know how to make the ADB adjust the accessibility focus while Talkback is on. I have tried:

adb shell input trackball roll 0 1
adb shell input [stylusdpad|keyboard|mouse|touchpad|gamepad|touchnavigation|joystick|touchscreen|stylus|trackball] swipe 180 780 540 780
adb shell input keyboard keyevent KEYCODE_TAB
adb shell input keyevent KEYCODE_NAVIGATE_NEXT
adb shell "input keyevent KEYCODE_ALT_LEFT & input keyevent KEYCODE_DPAD_LEFT"

I have also tried recording events using adb shell getevent and playing them back without success.

But I always have to physically swipe the screen (i.e. an ADB swipe does not work) to get the accessibility focus to change. Is there a way to do this with accessibility, just a next and previous movement?

I have found this article by Google:

Navigation

• Move to next item: Alt + Right arrow Note: In continuous reading mode, this shortcut fast-forwards through the text.
• Move to previous item: Alt + Left arrow Note: In continuous reading mode, this shortcut rewinds the text.

Which means I just need to send multiple key presses at once, right? I tried this, based on another SO answer:

device="/dev/input/event3"
ALT_KEY=57#18 #KEYCODE_ALT_LEFT
LEFT_KEY=21#37 #KEYCODE_DPAD_RIGHT
RIGHT_KEY=22#39 #KEYCODE_DPAD_RIGHT

device="/dev/input/event0"
adb shell "sendevent $device 1 $ALT_KEY 1 & sendevent $device 0 0 0 & sendevent $device 1 $RIGHT_KEY 1 & sendevent $device 0 0 0"
device="/dev/input/event1"
adb shell "sendevent $device 1 $ALT_KEY 1 & sendevent $device 0 0 0 & sendevent $device 1 $RIGHT_KEY 1 & sendevent $device 0 0 0"
device="/dev/input/event2"
adb shell "sendevent $device 1 $ALT_KEY 1 & sendevent $device 0 0 0 & sendevent $device 1 $RIGHT_KEY 1 & sendevent $device 0 0 0"
device="/dev/input/event3"
adb shell "sendevent $device 1 $ALT_KEY 1 & sendevent $device 0 0 0 & sendevent $device 1 $RIGHT_KEY 1 & sendevent $device 0 0 0"

device="/dev/input/event0"
adb shell "sendevent $device 1 $ALT_KEY 1 && sendevent $device 0 0 0 && sendevent $device 1 $RIGHT_KEY 1 && sendevent $device 0 0 0"
device="/dev/input/event1"
adb shell "sendevent $device 1 $ALT_KEY 1 && sendevent $device 0 0 0 && sendevent $device 1 $RIGHT_KEY 1 && sendevent $device 0 0 0"
device="/dev/input/event2"
adb shell "sendevent $device 1 $ALT_KEY 1 && sendevent $device 0 0 0 && sendevent $device 1 $RIGHT_KEY 1 && sendevent $device 0 0 0"
device="/dev/input/event3"
adb shell "sendevent $device 1 $ALT_KEY 1 && sendevent $device 0 0 0 && sendevent $device 1 $RIGHT_KEY 1 && sendevent $device 0 0 0"

device="/dev/input/event0"
adb shell "sendevent $device 1 $ALT_KEY 1 && sendevent $device 1 $RIGHT_KEY 1 && sendevent $device 0 0 0"
device="/dev/input/event1"
adb shell "sendevent $device 1 $ALT_KEY 1 && sendevent $device 1 $RIGHT_KEY 1 && sendevent $device 0 0 0"
device="/dev/input/event2"
adb shell "sendevent $device 1 $ALT_KEY 1 && sendevent $device 1 $RIGHT_KEY 1 && sendevent $device 0 0 0"
device="/dev/input/event3"
adb shell "sendevent $device 1 $ALT_KEY 1 && sendevent $device 1 $RIGHT_KEY 1 && sendevent $device 0 0 0"

(even though I know device0 is actually the keyboard device, I wanted to try them all)

Answer 1

My answer here is going to be as succinct as possible. My full code is available on GitHub.

As far as I am aware, a developer cannot perform an accessibility action via ADB, they would have to create an Accessibility service in order to act on behalf of an Accessibility user and create a Broadcast Receiver so that it can take input via the ADB. This is two thirds of the answer, requiring one more component as developers cannot activate accessibility services via the ADB! This has to be done manually each time accessibility is toggled or through accessibility shortcuts - of which there can be only one. EDIT I figured this bit out as well :)

Accessibility Service

The developer documentation provides a mechanism for an Accessibility Service:

An accessibility service is an application that provides user interface enhancements to assist users with disabilities, or who may temporarily be unable to fully interact with a device. For example, users who are driving, taking care of a young child or attending a very loud party might need additional or alternative interface feedback.

I followed the Google Codelab to construct a service that could take actions on the part of a user. Here is a snippet from the service, for swiping left and right (user navigation):

    fun swipeHorizontal(leftToRight: Boolean) {
        val swipePath = Path()
        if (leftToRight) {
            swipePath.moveTo(halfWidth - quarterWidth, halfHeight)
            swipePath.lineTo(halfWidth + quarterWidth, halfHeight)
        } else {
            swipePath.moveTo(halfWidth + quarterWidth, halfHeight)
            swipePath.lineTo(halfWidth - quarterWidth, halfHeight)
        }
        val gestureBuilder = GestureDescription.Builder()
        gestureBuilder.addStroke(StrokeDescription(swipePath, 0, 500))
        dispatchGesture(gestureBuilder.build(), GestureResultCallback(baseContext), null)
    }

Broadcast receiver.

It's important to note that the Receiver is registered when the service is enabled:

    override fun onServiceConnected() {
        IntentFilter().apply {
            addAction(ACCESSIBILITY_CONTROL_BROADCAST_ACTION)

            priority = 100

            registerReceiver(accessibilityActionReceiver, this)           
        }
   // REMEMBER TO DEREGISTER!

Then the receiver can respond to intents and invoke the service:

    override fun onReceive(context: Context?, intent: Intent?) {
        require(intent != null) { "Intent is required" }
        val serviceReference = //get a reference to the service somehow

        intent.getStringExtra(ACCESSIBILITY_ACTION)?.let {
            serviceReference.apply {
                when (it) {
                    ACTION_NEXT -> swipeHorizontal(true)
                    ACTION_PREV -> swipeHorizontal(false)
                    //...

example usage on the adb:

adb shell am broadcast -a com.balsdon.talkback.accessibility -e ACTION "ACTION_PREV"

EDIT

Starting the service via adb:

Thanks to this comment

TALKBACK="com.google.android.marvin.talkback/com.google.android.marvin.talkback.TalkBackService"
ALLYDEV="com.balsdon.accessibilityBroadcastService/.AccessibilityBroadcastService"
adb shell settings put secure enabled_accessibility_services $TALKBACK:$ALLYDEV

Accessibility shortcut

** EDIT: NO LONGER NEEDED BUT I'LL LEAVE IT HERE ANYWAY **

Possibly the most annoying thing is that every time accessibility is toggled, the service is turned off. So I added my service as a shortcut with the VOLUME_UP and VOLUME_DOWN keys pressed. Thanks to this question.

  INPUT_DEVICE="/dev/input/event1" # VOLUME KEYS EVENT FILE
  VOLUME_DOWN=114 #0x0072
  VOLUME_UP=115   #0x0073
  BLANK_EVENT="sendevent $INPUT_DEVICE 0 0 0"

  INST_DN="sendevent $INPUT_DEVICE 1 $VOLUME_DOWN 1 && $BLANK_EVENT && sendevent $INPUT_DEVICE 1 $VOLUME_UP 1 && $BLANK_EVENT"
  INST_UP="sendevent $INPUT_DEVICE 1 $VOLUME_DOWN 0 && $BLANK_EVENT && sendevent $INPUT_DEVICE 1 $VOLUME_UP 0 && $BLANK_EVENT"
  adb shell "$INST_DN"
  sleep 3
  adb shell "$INST_UP"

I already have a script for toggling accessibility on/off, so I just tacked this on top of that and I get my service running every time.

EDIT 2

I put an issue on AOSP regarding the fact that a developer needs to write a "service that swipes" as opposed to a "service that navigates". This is problematic as the gestures can be modified and then my system will not behave as expected. Instead I should be able to call the particular action I want to do NAVIGATE TO NEXT ELEMENT or NAVIGATE TO PREVIOUS ELEMENT as opposed to "SWIPE RIGHT" and "SWIPE LEFT" - having read the WCAG guidelines I do not believe that this is violating the principles.