Cropping a Perspective Transformation of Image on Android

Question

I'm trying to do a perspective transform on a bitmap that I capture via the camera. The user adjusts a bounding quad (as depicted by the white box) around a rectangular object. I then attempt to transform this to a rectangular image using the code below:

public static Bitmap perspectiveTransformation(Bitmap bitmap,BoundingQuad boundingQuad)
{
    Matrix matrix = new Matrix();
    float[] dst = new float[] {
            0,
            0,
            bitmap.getWidth(),
            0,
            bitmap.getWidth(),
            bitmap.getHeight(),
            0,
            bitmap.getHeight()
    };
    float[] src = new float[] {
            boundingQuad.topLeft.x,
            boundingQuad.topLeft.y,
            boundingQuad.topRight.x,
            boundingQuad.topRight.y,
            boundingQuad.bottomRight.x,
            boundingQuad.bottomRight.y,
            boundingQuad.bottomLeft.x,
            boundingQuad.bottomLeft.y
    };
    matrix.setPolyToPoly(src, 0, dst, 0, src.length >> 1);
    return Bitmap.createBitmap(bitmap, 0, 0, bitmap.getWidth(), bitmap.getHeight(), matrix, true);
}

However my resulting image contains image data that is outside the boundary of my quad. This would be acceptable if I could figure out what the coordinates of the quad are after transformation so I could crop the result but I have absolutely no idea how to do that.

Any help would be greatly appreciated in either finding the coordinates of the quad after transformation or ideally finding a way to prevent this situation from occurring in the first place.

Input:

http://i.stack.imgur.com/33RfN.png

Output:

http://i.stack.imgur.com/zWFvA.png

Answer 1

I had the same problem and solved it by finding the coordinates of the rectangle after the transformation.

To find these coordinates you have to understand what's going on. The matrix defines a perspective transformation, which is given by the 4 edge-points of the quad and the corresponding points. You have done this with the following code:

Matrix matrix = new Matrix();
float[] dst = new float[] {
        0,
        0,
        bitmap.getWidth(),
        0,
        bitmap.getWidth(),
        bitmap.getHeight(),
        0,
        bitmap.getHeight()
};
float[] src = new float[] {
        boundingQuad.topLeft.x,
        boundingQuad.topLeft.y,
        boundingQuad.topRight.x,
        boundingQuad.topRight.y,
        boundingQuad.bottomRight.x,
        boundingQuad.bottomRight.y,
        boundingQuad.bottomLeft.x,
        boundingQuad.bottomLeft.y
}; 
matrix.setPolyToPoly(src, 0, dst, 0, src.length >> 1);

This means (for example) that the top-left point of your quad is transformed to the point (0,0). You can verify this by applying the matrix to the points and check the resulting values. To apply the matrix you can use the method mapPoints(...). The defined transformation-matrix works fine. The (at the first look) strange behavior of this transformation results from the creation of the bitmap:

    return Bitmap.createBitmap(bitmap, 0, 0, bitmap.getWidth(), bitmap.getHeight(), matrix, true);

The resulting bitmap seems to be wrong, because some points (for example all points to the left of the top-left point of the quad) are transformed to negative coordinates and if you want to draw something into a bitmap the coordinates have to be positive. Because of this the transformed points are shift and that leads to strange coordinates of the transformed points in the bitmap.

To conclude: The points are transformed correctly to the new coordinates, but can not be displayed and because of this they are shifted and the shifted coordinates of the transformed points in the bitmap are not the coordinates, which are defined in the transformation-matrix.

To solve this problem and get the proper coordinates of the transformed points in the bitmap, you have to calculate the values of the shift. The shift consists of a x-value and a y-value.

To calculate the x-value I transformed the x-value of the top-left point and the x-value of the lower-left point of the original image with the previously defined matrix. Either the top-left point or the lower-left point is transformed to the left boundary of the resulting bitmap and because of this the x-value of the bitmap-coordinate of this point is equal to 0 and the negated (because the x-value needs to be positive) x-value of the transformed coordinates is the x-value of the shift. The point, which is transformed to the left boundary of the resulting bitmap is the point with the maximal negated x-value. Therefore the x-value of the shift is the maximum of the negated x-values of the transformed top-left and lower-left point.

To calculate the y-value I transformed the y-value of the top-left point and the y-value of the top-right point of the original image, because this are the possible points which are transformed to the top boundary of the resulting bitmap and the y-value of the transformed point is equal to 0 in the resulting bitmap. By taking again the maximum of the negated values of the transformed y-values, you get the y-value of the shift.

The resulting code looks like this:

    float[] mappedTL = new float[] { 0, 0 };
    matrix.mapPoints(mappedTL);
    int maptlx = Math.round(mappedTL[0]);
    int maptly = Math.round(mappedTL[1]);

    float[] mappedTR = new float[] { bitmap.getWidth(), 0 };
    matrix.mapPoints(mappedTR);
    int maptrx = Math.round(mappedTR[0]);
    int maptry = Math.round(mappedTR[1]);

    float[] mappedLL = new float[] { 0, bitmap.getHeight() };
    matrix.mapPoints(mappedLL);
    int mapllx = Math.round(mappedLL[0]);
    int maplly = Math.round(mappedLL[1]);

    int shiftX = Math.max(-maptlx, -mapllx);
    int shiftY = Math.max(-maptry, -maptly);

    Bitmap resultBitmap = Bitmap.createBitmap(bitmap, 0, 0, bitmap.getWidth(), bitmap.getHeight(), matrix, true);
    return Bitmap.createBitmap(resultBitmap, shiftX, shiftY, bitmap.getWidth(), bitmap.getHeight(), null, true);