UiAutomator getLastTraversedText()

Question

I was trying to test an Android Webview using Android UiAutomator. As I understand the documentation, scrolling through a WebvView would generate UI traversal events, and those should be readable via getUiDevice().getLastTraversedText().

However, when I use getUiDevice().pressDPadDown() to scroll through a web view, getUiDevice().getLastTraversedText() keeps returning null.

What have I missed?

If anyone has gotten this call to work I would very much appreciate a short code sample.

Answer 1

Bad news: I've spent several hours trying to fathom out how to get this working, however I've yet to get anything other than null in response to the calls to getUiDevice().getLastTraversedText().

FYI here are the things I've tried and discovered:

• running uiautomator events in an adb shell should dump all the Accessibility Events. It certainly reports various events, however it's virtually silent when I manually scroll through the contents of a WebView. If the content in the WebView is scrolled e.g. after navigating up or down beyond what's displayed on screen I get the following type of event reported:

03-10 19:44:47.436 EventType: TYPE_VIEW_SCROLLED; EventTime: 911700; PackageName: com.example.simplewebview; MovementGranularity: 0; Action: 0 [ ClassName: android.webkit.WebView; Text: []; ContentDescription: null; ItemCount: -1; CurrentItemIndex: -1; IsEnabled: true; IsPassword: false; IsChecked: false; IsFullScreen: false; Scrollable: true; BeforeText: null; FromIndex: -1; ToIndex: -1; ScrollX: 0; ScrollY: 270; MaxScrollX: 0; MaxScrollY: 544; AddedCount: -1; RemovedCount: -1; ParcelableData: null ]; recordCount: 0

• If I enable the Explore-by-touch accessibility setting then I cannot use uiautomator in adb shell. Each time I run a uiautomator command, I get a response of Killed. I was using Explore-By-Touch to speak the text that's displayed. While I wasn't able to navigate to all the links with Explore-By-Touch enabled (or when using the infuriating Virtual D-Pad which is part of the Android Accessibility Suite) it did read out the LinkText for some of the links (e.g. it skipped the Japanese characters on the page I was using for testing of http://pickalize.info/ )

• If I run a script similar to yours (for a simple WebView app I created for the purpose) then I can see the UI in the WebView highlight various elements, in my case web links from the following web page I was investigating for other reasons at the time http://pickalize.info/ The background of the currently selected link goes to pale blue, however no text is returned by the call to getUiDevice().getLastTraversedText()

My best guess is that we're either trying to use the method inappropriately. Perhaps we're supposed to request and parse the DOM of the WebView (goodness knows how), but see the following comment from the documentation of getLastTraversedText()

When the view control used can return a reference to is Document Object Model, it is recommended then to use the view's DOM instead.

BTW: I'm testing with Android 4.2.2 on physical devices.

Good luck with your investigations. I'll keep looking at the problem from time to time to help me learn more about the appropriateness and capabilities of Ui Automator.

Update (26 March 2013): I tested again with 'Enhance web accessibility' set to 'Allowed' in the device's Accessibility Setting menu. I still get null returned by getLastTraversedText()

Here's the guts of my test code:

public class WebViewNavigator extends UiAutomatorTestCase {

public void testNavigationDownGeneratesEventText() throws UiObjectNotFoundException {
    getUiDevice().pressHome();
    UiObject allAppsButton = new UiObject(new UiSelector().description("Apps"));
    allAppsButton.clickAndWaitForNewWindow();

    UiObject appsTab = new UiObject(new UiSelector().text("Apps"));
    appsTab.click();

    UiScrollable appViews = new UiScrollable(new UiSelector().scrollable(true));
    appViews.setAsHorizontalList();

    String nameOfAppToLaunch = "SimpleWebView";
    UiAutomatorHelpers.launchAppCalled(nameOfAppToLaunch);

    UiObject simpleWebViewValidation = new UiObject(new UiSelector().packageName("com.example.simplewebview"));
    assertTrue("The package name seems incorrect for the app we want to launch", simpleWebViewValidation.exists());
    simpleWebViewValidation.clickAndWaitForNewWindow();

    // Ok we should have started the app now... On to the test

    Log.i("WebViewNavigatorTest", "About to start navigating down the contents of the webview");
    int closeToInfinity = 10;
    for (int i = 0; i < closeToInfinity; i++) {
        boolean goneDown = getUiDevice().pressDPadDown();
        if (!goneDown) {
            break;
        }

        String lastTraversedText = getUiDevice().getLastTraversedText();
        if (lastTraversedText != null) {
            Log.i("WebViewNavigatorTest", lastTraversedText);
        } else {
            Log.w("WebViewNavigatorTest", "(null) returned for getLastTraversedText()");
        }
    }   
}

}

And here's the adb command I use to see the messages when this code is executed on the Android device: adb logcat WebViewNavigatorTest:I *:S

Followed by the output:

I/WebViewNavigatorTest(29358): About to start navigating down the contents of the webview
W/WebViewNavigatorTest(29358): (null) returned for getLastTraversedText()
W/WebViewNavigatorTest(29358): (null) returned for getLastTraversedText()
W/WebViewNavigatorTest(29358): (null) returned for getLastTraversedText()
W/WebViewNavigatorTest(29358): (null) returned for getLastTraversedText()
W/WebViewNavigatorTest(29358): (null) returned for getLastTraversedText()
W/WebViewNavigatorTest(29358): (null) returned for getLastTraversedText()
W/WebViewNavigatorTest(29358): (null) returned for getLastTraversedText()
W/WebViewNavigatorTest(29358): (null) returned for getLastTraversedText()
W/WebViewNavigatorTest(29358): (null) returned for getLastTraversedText()
W/WebViewNavigatorTest(29358): (null) returned for getLastTraversedText()