How to press a button twice using Google UiAutomator?

Question

I have the following script for typing '33' into the Calculator, in Android, using UiAutomator. However, only the first '3' is accepted, the second press is entirely ignored.

import com.android.uiautomator.core.*;
import com.android.uiautomator.testrunner.UiAutomatorTestCase;

public class MyFirstUiAutomatorTest extends UiAutomatorTestCase {
    UiObject getByDescription(String description) {
        return new UiObject(new UiSelector().description(description));
    }

    UiObject getByText(String description) {
        return new UiObject(new UiSelector().text(description));
    }

    UiObject scrollableGetByText(String text ) throws UiObjectNotFoundException {
            UiScrollable uiScrollable = new UiScrollable(new UiSelector().scrollable(true));
            uiScrollable.setAsHorizontalList();
            return uiScrollable.getChildByText(new UiSelector().className(
                    android.widget.TextView.class.getName()),
                    text);      
    }

    public void testStuff() throws UiObjectNotFoundException {
        getUiDevice().pressHome();
        getByDescription("Apps").clickAndWaitForNewWindow();
        getByText("Apps").click();
        scrollableGetByText("Calculator").clickAndWaitForNewWindow();

        // pressing '+' and '=' effectively clears the previous input
        getByText("+").click();
        getByText("=").click();
        getByText("3").click();
        // this second '3' is ignored
        getByText("3").click();
    }
}

I've tried adding a sleep for 2 seconds after the first click, by doing:

        try {
            Thread.sleep(2000);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }

... but that didn't change anything.

I also tried clicking on a different button, in between the 2 '3's, ie:

        new UiObject(new UiSelector().text("3")).click();
        new UiObject(new UiSelector().className("android.widget.EditText")).click();
        new UiObject(new UiSelector().text("3")).click();

... but that didn't work either.

Ideas?

(Note: using Android 4.1.2, in an AVD; running on Ubuntu linux 12.04)

Edit, following Rami's observations, I tried the following, to reuse the same UiObject object for a second request for the same description:

HashMap<String,UiObject> objectByText = new HashMap<String,UiObject>(); 
UiObject getByText(String description) {
    if( objectByText.containsKey(description)) {
        System.out.println("" + objectByText.get(description) );
        return objectByText.get(description);
    }
    System.out.println("Created new object for [" + description + "]");
    UiObject object = new UiObject(new UiSelector().text(description));
    objectByText.put(description, object );
    System.out.println("" + object );
    return object;
}

... but it didn't work, even though it is clearly reusing the same UiObject each time, because it only says 'Created new object for [3]' once.

Then I tried the UiDevice.click() 'trick', by creating a function 'click' as follows, again, following Rami's observations:

void click(UiObject target ) throws UiObjectNotFoundException {
    Rect rect = target.getBounds();
    System.out.println("rect: " + rect );
    getUiDevice().click(rect.centerX(), rect.centerY());
}

However, this didn't work for me either: only the first '3' appears, and the second is ignored, even though both clicks are clearly in the same place, because the rect: output locations are identical. If I click '3' twice manually, using my own desktop mouse, then both 3s appear ok.

I also tried adding a two second Thread.sleep() between the clicks, and still only a single '3' appeared for me.

Answer 1

Instead of just searching by text, try searching by the text and the class. Here is a sample method for doing so.

Uiobject getByTextAndClass(String text, String className) {
    return new UiObject(new UiSelector().text(text).className(className));
}

And then if you are trying to call this method for the Calculator button with number 3 on it:

getByTextAndClass("3", android.widget.Button.class.getName()).click();

You can use the UiAutomatorViewer tool: {android-sdk}/tools/uiautomator.bat to check the classnames and other attributes of different UiObjects.

This works on my 4.2.2 devices, but I am downloading 4.1.2 to test it on there as well.

I tried it on a 4.1.2 AVD and it works on my Windows machine.

Answer 2

This issue seems to be built into the way UiObject is set up. As far as I can tell, an instance of UiObject doesn't seem to refer to a specific object on the screen. It's more like an instance of what's pointed to by the UiSelector when the UiObject was defined. The key here is that each time the object is called, it searches for a new item that fits the UiSelector requirements.

Evidence:

UiObject ok = new UiObject(new UiSelector().text("OK"));
ok.click();
// clicks a button with the text ok on the page
// Navigate to some different page with a button with the text ok
ok.click();
// this ok button on a completely different page is still clicked

Therefore, I think Rami's workaround might actually by the correct way to do it (store the location of the object through its coordinates and click on the center of that location). Important thing to note about Rami's workaround, save the coordinates/rectangle once and then reuse the coordinates/rectangle whenever you want to refer to the object. Using obj.getBounds() multiple times on the same object doesn't seem to work.

Also, records of the items that have already been passed over during a UiObject instantiation seem to be stored in the app. Running a test twice that finds a UiObject (ex. button 4 in the set alarm page) and clicks it fails the same way as a test that finds a UiObject and clicks it twice. This means that data of object queries must be stored on the device because resetting UIautomator makes no difference.

Answer 3

I have the same problem, looks like its looking for another UI element that have the same text. I'm trying to automate the creation of an Alarm but when the time is at for example 4:45 pm. Uiautomator will click the right Button.

 clickByText("4");
 clickByText("4");
 clickByText("5");
 clickByText("PM");
 clickByText("OK");

private void clickByText(String text) throws UiObjectNotFoundException {
            UiObject obj = new UiObject(new UiSelector().text(text));
            obj.click();
    }

Here is the Log for the first "4" click:

07-04 16:54:05.259: I/QueryController(25605): Matched selector: UiSelector[TEXT=4] <<==>> [android.view.accessibility.AccessibilityNodeInfo@592f1; boundsInParent: Rect(0, 0 - 202, 129); boundsInScreen: Rect(56, 508 - 258, 637); packageName: com.android.deskclock; className: android.widget.Button; text: 4; contentDescription: null; checkable: false; checked: false; focusable: true; focused: false; selected: false; clickable: true; longClickable: false; enabled: true; password: false; scrollable: false; [ACTION_FOCUS, ACTION_SELECT, ACTION_CLEAR_SELECTION, ACTION_CLICK, ACTION_ACCESSIBILITY_FOCUS, ACTION_NEXT_AT_MOVEMENT_GRANULARITY, ACTION_PREVIOUS_AT_MOVEMENT_GRANULARITY]]

Here is the Log for the second "4" click:

07-04 16:54:07.798: I/QueryController(25605): Matched selector: UiSelector[TEXT=4] <<==>> [android.view.accessibility.AccessibilityNodeInfo@5bffd; boundsInParent: Rect(0, 0 - 66, 90); boundsInScreen: Rect(191, 264 - 257, 354); packageName: com.android.deskclock; className: android.widget.TextView; text: 4; contentDescription: null; checkable: false; checked: false; focusable: false; focused: false; selected: false; clickable: false; longClickable: false; enabled: true; password: false; scrollable: false; [ACTION_SELECT, ACTION_CLEAR_SELECTION, ACTION_ACCESSIBILITY_FOCUS, ACTION_NEXT_AT_MOVEMENT_GRANULARITY, ACTION_PREVIOUS_AT_MOVEMENT_GRANULARITY]]

We can see that one is clicking the right android.widget.Button and the second click its looking for android.widget.TextView.