How to generate zoom/pinch gesture for testing for Android

Question

I'm trying to generate zoom/pinch gesture for testing, I have tried to use the Android API MotionEvent.obtain(), but found it is hard to implement the zoom/pinch events.

I referenced the API from MotionEvent.obtain. Can you tell me the correct method?

Here is my implementation:

//for zoom, we need four points coordinations: start0, start1, end0, end1

Instrumentation inst;

// action down event
MotionEvent event = MotionEvent.obtain(downTime, eventTime, MotionEvent.ACTION_DOWN, 1, prop_start0, pointerCoords_start0, 0,  0, 0, 0, 0, 0, 0, 0 );
inst.sendPointerSync(event);

// action pointer 2 down event
event = MotionEvent.obtain(downTime, eventTime, MotionEvent.ACTION_POINTER_2_DOWN, 2, properties_start0_1, pointerCoords_start0_1, 0, 0, 0, 0, 0, 0, 0, 0);
inst.sendPointerSync(event);

// action move events
duration = 1000; //1000 ms
event_interval = 10;//time interval between consecutive events 10ms 
moveEventNum = duration / event_interval; 
stepx0 = (end0.x - start0.x)/moveEventNum;
stepy0 = (end0.y - start0.y)/moveEventNum;
stepx1 = (end1.x - start1.x)/moveEventNum;
stepy1 = (end1.y - start1.y)/moveEventNum;

move_event0= start0;
move_event1 = start1;
for ( int i = 0; i < moveEventNum; i++) {
       //  [generate middle points here ]
       mov_event0.x += stepx0;
       mov_event0.y += stepy0;
       mov_event1.x += stepx1;
       mov_event1.y += stepy1;

      eventTime += event_interval;

      event = MotionEvent.obtain(downTime, eventTime, MotionEvent.ACTION_MOVE, 2, properties_move_event, pointerCoords_move_event0_1, 0, 0, 0, 0, 0, 0, 0, 0); 

     inst.sendPointerSync(event);
}

event = MotionEvent.obtain(downTime, eventTime, MotionEvent.ACTION_POINTER_2_UP, 2, properties_end0_1, pointerCoords_end0_1, 0, 0, 0, 0, 0, 0, 0, 0);
inst.sendPointerSync(event);

event = MotionEvent.obtain(downTime, eventTime, MotionEvent.ACTION_UP, 1, end1, pointerCoords_end1, 0, 0, 0, 0, 0, 0, 0, 0 );
inst.sendPointerSync(event);

Answer 1

well, i have found the issue.

ISSUE:

When using the obtain() API, we have to set the pressure and size of the points in each event.

For

obtain(long, long, int, int, android.view.MotionEvent.PointerProperties[], 
android.view.MotionEvent.PointerCoords[], int, int, float, float, int, int, int, int)

the PointerCoords[], we have to set the pressure and size to 1, the default values are 0.

For

public static MotionEvent obtain (long downTime, long eventTime, int action,
         float x, float y, int metaState)

     Create a new MotionEvent, filling in a subset of the basic motion values. 
    Those not specified here are: device id (always 0), pressure and size (always 1), 
x and y precision (always 1), and edgeFlags (always 0).

since the default pressure and size are 1, so we don't need to set them.

My tips for creating gestures:

1. following the real gesture sequence, since we want to simulate the real gestures

• override the onTouchEvent() to check the real events received by application. These events can also be used for comparison of real user touch events and generated touch events Take browser for example:

  a) @Override public boolean onTouchEvent(MotionEvent event) { Log.i("WebView", event.toString() + event.getAction()); boolean rt = super.onTouchEvent(event); return rt; }

• manually touch screen to get the real gesture sequence from onTouchEvent() in a). We can follow the event sequence when generating events. -- If we don’t follow the gesture event sequence, the instrumented events may be rejected.

• Here is a valid event sequence of zoom gesture, (the downTime is the same for all the events)

  i. ACTION_DOWN of one start point

  ii. ACTION_POINTER_2_DOWN of two start points

  iii.ACTION_MOVE of two middle points

  iv. ACTION_POINTER_2_UP of two end points

  v. ACTION_UP of one end point

2. use the API MotionEvent.obtain correctly

• There are two most used obtain() API.

public static MotionEvent obtain (long downTime, long eventTime, int action, float x, float y, int metaState)

AND

public static MotionEvent obtain(long, long, int, int, android.view.MotionEvent.PointerProperties[], android.view.MotionEvent.PointerCoords[], int, int, float, float, int, int, int, int)

The first one is usually used for single point gestures, like fling, scroll, click etc. The parameters (pressure, size, xPresion, yPresion) for this function are all set to 1.

And the second one is a more general one, and can be used for multi-touch events generation. While for the second one, we have to set the pressure, size in pointerCoords of each touch point to 1.

Here is the example to generate the zoom gesture:

public static void generateZoomGesture(Instrumentation inst,
        long startTime, boolean ifMove, GestureInfo.Point startPoint1,
        GestureInfo.Point startPoint2, GestureInfo.Point endPoint1,
        GestureInfo.Point endPoint2, int duration) {

    if (inst == null || startPoint1 == null
            || (ifMove && endPoint1 == null)) {
        return;
    }

    long eventTime = startTime;
    long downTime = startTime;
    MotionEvent event;
    float eventX1, eventY1, eventX2, eventY2;

    eventX1 = startPoint1.x;
    eventY1 = startPoint1.y;
    eventX2 = startPoint2.x;
    eventY2 = startPoint2.y;

    // specify the property for the two touch points
    PointerProperties[] properties = new PointerProperties[2];
    PointerProperties pp1 = new PointerProperties();
    pp1.id = 0;
    pp1.toolType = MotionEvent.TOOL_TYPE_FINGER;
    PointerProperties pp2 = new PointerProperties();
    pp2.id = 1;
    pp2.toolType = MotionEvent.TOOL_TYPE_FINGER;

    properties[0] = pp1;
    properties[1] = pp2;

    //specify the coordinations of the two touch points
    //NOTE: you MUST set the pressure and size value, or it doesn't work
    PointerCoords[] pointerCoords = new PointerCoords[2];
    PointerCoords pc1 = new PointerCoords();
    pc1.x = eventX1;
    pc1.y = eventY1;
    pc1.pressure = 1;
    pc1.size = 1;
    PointerCoords pc2 = new PointerCoords();
    pc2.x = eventX2;
    pc2.y = eventY2;
    pc2.pressure = 1;
    pc2.size = 1;
    pointerCoords[0] = pc1;
    pointerCoords[1] = pc2;

    //////////////////////////////////////////////////////////////
    // events sequence of zoom gesture
    // 1. send ACTION_DOWN event of one start point
    // 2. send ACTION_POINTER_2_DOWN of two start points
    // 3. send ACTION_MOVE of two middle points
    // 4. repeat step 3 with updated middle points (x,y),
    //      until reach the end points
    // 5. send ACTION_POINTER_2_UP of two end points
    // 6. send ACTION_UP of one end point
    //////////////////////////////////////////////////////////////

    // step 1
    event = MotionEvent.obtain(downTime, eventTime, 
                MotionEvent.ACTION_DOWN, 1, properties, 
                pointerCoords, 0,  0, 1, 1, 0, 0, 0, 0 );

    inst.sendPointerSync(event);

    //step 2
    event = MotionEvent.obtain(downTime, eventTime, 
                MotionEvent.ACTION_POINTER_2_DOWN, 2, 
                properties, pointerCoords, 0, 0, 1, 1, 0, 0, 0, 0);

    inst.sendPointerSync(event);

    //step 3, 4
    if (ifMove) {
        int moveEventNumber = 1;
        moveEventNumber = duration / EVENT_MIN_INTERVAL;

        float stepX1, stepY1, stepX2, stepY2;

        stepX1 = (endPoint1.x - startPoint1.x) / moveEventNumber;
        stepY1 = (endPoint1.y - startPoint1.y) / moveEventNumber;
        stepX2 = (endPoint2.x - startPoint2.x) / moveEventNumber;
        stepY2 = (endPoint2.y - startPoint2.y) / moveEventNumber;

        for (int i = 0; i < moveEventNumber; i++) {
            // update the move events
            eventTime += EVENT_MIN_INTERVAL;
            eventX1 += stepX1;
            eventY1 += stepY1;
            eventX2 += stepX2;
            eventY2 += stepY2;

            pc1.x = eventX1;
            pc1.y = eventY1;
            pc2.x = eventX2;
            pc2.y = eventY2;

            pointerCoords[0] = pc1;
            pointerCoords[1] = pc2;

            event = MotionEvent.obtain(downTime, eventTime,
                        MotionEvent.ACTION_MOVE, 2, properties, 
                        pointerCoords, 0, 0, 1, 1, 0, 0, 0, 0);

            inst.sendPointerSync(event);
        }
    }

    //step 5
    pc1.x = endPoint1.x;
    pc1.y = endPoint1.y;
    pc2.x = endPoint2.x;
    pc2.y = endPoint2.y;
    pointerCoords[0] = pc1;
    pointerCoords[1] = pc2;

    eventTime += EVENT_MIN_INTERVAL;
    event = MotionEvent.obtain(downTime, eventTime,
                MotionEvent.ACTION_POINTER_2_UP, 2, properties, 
                pointerCoords, 0, 0, 1, 1, 0, 0, 0, 0);
    inst.sendPointerSync(event);

    // step 6
    eventTime += EVENT_MIN_INTERVAL;
    event = MotionEvent.obtain(downTime, eventTime, 
                MotionEvent.ACTION_UP, 1, properties, 
                pointerCoords, 0, 0, 1, 1, 0, 0, 0, 0 );
    inst.sendPointerSync(event);
}

Answer 2

As mentioned in a comment by Täg, this can be used with Espresso. Here is my code for doing this in Espresso, based on the very good answer from longchuan and the update form peetasan.

In the Espresso test:

// Pinch out (to zoom in):
onView(withId(R.id.MyViewId)).perform(pinchOut());

The helper methods, which includes a nice way to get the correct coordinates from the view, and also includes correct error handling for uiController.injectMotionEvent(event):

public static ViewAction pinchOut() {
    return new ViewAction() {
        @Override
        public Matcher<View> getConstraints() {
            return ViewMatchers.isEnabled();
        }

        @Override
        public String getDescription() {
            return "Pinch out";
        }

        @Override
        public void perform(UiController uiController, View view) {
            Point middlePosition = getCenterPoint(view);

            final int startDelta = 0; // How far from the center point each finger should start
            final int endDelta = 500; // How far from the center point each finger should end (note: Be sure to have this large enough so that the gesture is recognized!)

            Point startPoint1 = new Point(middlePosition.x - startDelta, middlePosition.y);
            Point startPoint2 = new Point(middlePosition.x + startDelta, middlePosition.y);
            Point endPoint1 = new Point(middlePosition.x - endDelta, middlePosition.y);
            Point endPoint2 = new Point(middlePosition.x + endDelta, middlePosition.y);

            performPinch(uiController, startPoint1, startPoint2, endPoint1, endPoint2);
        }
    };
}

public static ViewAction pinchIn() {
    return new ViewAction() {
        @Override
        public Matcher<View> getConstraints() {
            return ViewMatchers.isEnabled();
        }

        @Override
        public String getDescription() {
            return "Pinch in";
        }

        @Override
        public void perform(UiController uiController, View view) {
            Point middlePosition = getCenterPoint(view);

            final int startDelta = 500; // How far from the center point each finger should start (note: Be sure to have this large enough so that the gesture is recognized!)
            final int endDelta = 0; // How far from the center point each finger should end

            Point startPoint1 = new Point(middlePosition.x - startDelta, middlePosition.y);
            Point startPoint2 = new Point(middlePosition.x + startDelta, middlePosition.y);
            Point endPoint1 = new Point(middlePosition.x - endDelta, middlePosition.y);
            Point endPoint2 = new Point(middlePosition.x + endDelta, middlePosition.y);

            performPinch(uiController, startPoint1, startPoint2, endPoint1, endPoint2);
        }
    };
}

@NonNull
private static Point getCenterPoint(View view) {
    int[] locationOnScreen = new int[2];
    view.getLocationOnScreen(locationOnScreen);
    float viewHeight = view.getHeight() * view.getScaleY();
    float viewWidth = view.getWidth() * view.getScaleX();
    return new Point(
            (int) (locationOnScreen[0] + viewWidth / 2),
            (int) (locationOnScreen[1] + viewHeight / 2));
}

private static void performPinch(UiController uiController, Point startPoint1, Point startPoint2, Point endPoint1, Point endPoint2) {
    final int duration = 500;
    final long eventMinInterval = 10;
    final long startTime = SystemClock.uptimeMillis();
    long eventTime = startTime;
    MotionEvent event;
    float eventX1, eventY1, eventX2, eventY2;

    eventX1 = startPoint1.x;
    eventY1 = startPoint1.y;
    eventX2 = startPoint2.x;
    eventY2 = startPoint2.y;

    // Specify the property for the two touch points
    MotionEvent.PointerProperties[] properties = new MotionEvent.PointerProperties[2];
    MotionEvent.PointerProperties pp1 = new MotionEvent.PointerProperties();
    pp1.id = 0;
    pp1.toolType = MotionEvent.TOOL_TYPE_FINGER;
    MotionEvent.PointerProperties pp2 = new MotionEvent.PointerProperties();
    pp2.id = 1;
    pp2.toolType = MotionEvent.TOOL_TYPE_FINGER;

    properties[0] = pp1;
    properties[1] = pp2;

    // Specify the coordinations of the two touch points
    // NOTE: you MUST set the pressure and size value, or it doesn't work
    MotionEvent.PointerCoords[] pointerCoords = new MotionEvent.PointerCoords[2];
    MotionEvent.PointerCoords pc1 = new MotionEvent.PointerCoords();
    pc1.x = eventX1;
    pc1.y = eventY1;
    pc1.pressure = 1;
    pc1.size = 1;
    MotionEvent.PointerCoords pc2 = new MotionEvent.PointerCoords();
    pc2.x = eventX2;
    pc2.y = eventY2;
    pc2.pressure = 1;
    pc2.size = 1;
    pointerCoords[0] = pc1;
    pointerCoords[1] = pc2;

    /*
     * Events sequence of zoom gesture:
     *
     * 1. Send ACTION_DOWN event of one start point
     * 2. Send ACTION_POINTER_DOWN of two start points
     * 3. Send ACTION_MOVE of two middle points
     * 4. Repeat step 3 with updated middle points (x,y), until reach the end points
     * 5. Send ACTION_POINTER_UP of two end points
     * 6. Send ACTION_UP of one end point
     */

    try {
        // Step 1
        event = MotionEvent.obtain(startTime, eventTime,
                MotionEvent.ACTION_DOWN, 1, properties,
                pointerCoords, 0, 0, 1, 1, 0, 0, 0, 0);
        injectMotionEventToUiController(uiController, event);

        // Step 2
        event = MotionEvent.obtain(startTime, eventTime,
                MotionEvent.ACTION_POINTER_DOWN + (pp2.id << MotionEvent.ACTION_POINTER_INDEX_SHIFT), 2,
                properties, pointerCoords, 0, 0, 1, 1, 0, 0, 0, 0);
        injectMotionEventToUiController(uiController, event);

        // Step 3, 4
        long moveEventNumber = duration / eventMinInterval;

        float stepX1, stepY1, stepX2, stepY2;

        stepX1 = (endPoint1.x - startPoint1.x) / moveEventNumber;
        stepY1 = (endPoint1.y - startPoint1.y) / moveEventNumber;
        stepX2 = (endPoint2.x - startPoint2.x) / moveEventNumber;
        stepY2 = (endPoint2.y - startPoint2.y) / moveEventNumber;

        for (int i = 0; i < moveEventNumber; i++) {
            // Update the move events
            eventTime += eventMinInterval;
            eventX1 += stepX1;
            eventY1 += stepY1;
            eventX2 += stepX2;
            eventY2 += stepY2;

            pc1.x = eventX1;
            pc1.y = eventY1;
            pc2.x = eventX2;
            pc2.y = eventY2;

            pointerCoords[0] = pc1;
            pointerCoords[1] = pc2;

            event = MotionEvent.obtain(startTime, eventTime,
                    MotionEvent.ACTION_MOVE, 2, properties,
                    pointerCoords, 0, 0, 1, 1, 0, 0, 0, 0);
            injectMotionEventToUiController(uiController, event);
        }

        // Step 5
        pc1.x = endPoint1.x;
        pc1.y = endPoint1.y;
        pc2.x = endPoint2.x;
        pc2.y = endPoint2.y;
        pointerCoords[0] = pc1;
        pointerCoords[1] = pc2;

        eventTime += eventMinInterval;
        event = MotionEvent.obtain(startTime, eventTime,
                MotionEvent.ACTION_POINTER_UP + (pp2.id << MotionEvent.ACTION_POINTER_INDEX_SHIFT), 2, properties,
                pointerCoords, 0, 0, 1, 1, 0, 0, 0, 0);
        injectMotionEventToUiController(uiController, event);

        // Step 6
        eventTime += eventMinInterval;
        event = MotionEvent.obtain(startTime, eventTime,
                MotionEvent.ACTION_UP, 1, properties,
                pointerCoords, 0, 0, 1, 1, 0, 0, 0, 0);
        injectMotionEventToUiController(uiController, event);
    } catch (InjectEventSecurityException e) {
        throw new RuntimeException("Could not perform pinch", e);
    }
}

/**
 * Safely call uiController.injectMotionEvent(event): Detect any error and "convert" it to an
 * IllegalStateException
 */
private static void injectMotionEventToUiController(UiController uiController, MotionEvent event) throws InjectEventSecurityException {
    boolean injectEventSucceeded = uiController.injectMotionEvent(event);
    if (!injectEventSucceeded) {
        throw new IllegalStateException("Error performing event " + event);
    }
}

Answer 3

In order to make the above code work with non-deprecated constants I used MotionEvent.ACTION_POINTER_DOWN + (pp2.id << MotionEvent.ACTION_POINTER_INDEX_SHIFT) instead of MotionEvent.ACTION_POINTER_2_DOWN

So I had to replace this line:

event = MotionEvent.obtain(downTime, eventTime, MotionEvent.ACTION_POINTER_2_DOWN, 2, properties, pointerCoords, 0, 0, 1, 1, 0, 0, 0, 0);

with

event = MotionEvent.obtain(downTime, eventTime, MotionEvent.ACTION_POINTER_DOWN + (pp2.id << MotionEvent.ACTION_POINTER_INDEX_SHIFT), 2, properties, pointerCoords, 0, 0, 1, 1, 0, 0, 0, 0);