Android accessibility: Talkback traverses elements in wrong order

Question

I have a layout with a toolbar and a view that will host other controls:

<FrameLayout
    android:layout_width="match_parent"
    android:layout_height="match_parent">

    <Toolbar
        android:id="@+id/toolbar"
        android:layout_width="match_parent"
        android:layout_height="wrap_content" />

    <ViewStub
        android:id="@+id/root_view_stub"
        android:layout_width="match_parent"
        android:layout_height="match_parent" />
</FrameLayout>

A FrameLayout is used so that the toolbar, which has a positive "elevation", can be translucent and the view extends to under the toolbar. The toolbar and the view have the same top position but different height. When Talkback builds the view hierarchy, it puts the toolbar at the bottom even though it's defined first. "accessibilityTraversalBefore" and "accessibilityTraversalAfter" on the views have no effect. The only solution I found so far was to add a top margin of 1px or 0.1px to the ViewStub. This probably has something to do with the code here:

https://github.com/google/talkback/blob/master/src/main/java/com/android/utils/traversal/ReorderedChildrenIterator.java

The code tries to reorder the nodes based on their screen positions. Adding the top margin is a bit hacky and may break some day. Is there a better solution?

Answer 1

I figured out what's going on after debugging my app and TalkBack app. The accessibility traversal order is not controlled by the order views are defined in the layout. Adding accessibilityTraversalBefore or accessibilityTraversalAfter to the toolbar or ViewStub in activity_react_native.xml has no effect. I couldn't find any documentation on traversal order so I checked out the code. Here is how TalkBack works:

When TalkBack is turned on, all one-finger gestures are sent to TalkBack instead of to the app. Those gestures can be redefined in TalkBack settings. After a swiping up and down gesture, TalkBack first enumerates all focusable views on the current visible page. This is done by sending a message to the app. In the app process, the accessibility view hierarchy is built by iterating through all child views recursively. It's done in ViewGroup code: https://github.com/aosp-mirror/platform_frameworks_base/blob/d18ed49f9dba09b85782c83999a9103dec015bf2/core/java/android/view/ViewGroup.java#L2314. The last argument of this call is a hardcoded true which sorts child views based on their positions on the screen. The comparison logic is here: https://github.com/aosp-mirror/platform_frameworks_base/blob/d18ed49f9dba09b85782c83999a9103dec015bf2/core/java/android/view/ViewGroup.java#L8397. When two views have the same top, left and width, the one with bigger height comes before the one with smaller height or area, which seems like a bad choice in most scenarios. After the tree is built, it's sent back to TalkBack, which then reorders the elements based on the traversalBefore or traversalAfter properties. Since the Toolbar and ViewStub aren't focusable elements, they don't appear in this tree, which is why those two properties don't work.

The fix is to add android:importantForAccessibility to both toolbar and ViewStub. This will add them into the accessibility tree from the ViewGroup, so that TalkBack can act on accessibilityTraversalBefore/After attributes.

There is also an workaround that moves the toolbar upward by 1px then adds 1px padding inside. That extra 1px should be clipped by FrameLayout so there is no visual change:

android:layout_marginTop="-1px"
android:paddingTop="1px"