OpenCV Python cv2.perspectiveTransform

Question

I'm currently trying to video stabilization using OpenCV and Python. I use the following function to calculate rotation:

def accumulate_rotation(src, theta_x, theta_y, theta_z, timestamps, prev, current, f, gyro_delay=None, gyro_drift=None, shutter_duration=None):
    if prev == current:
        return src

    pts = []
    pts_transformed = []
    for x in range(10):
        current_row = []
        current_row_transformed = []
        pixel_x = x * (src.shape[1] / 10)
        for y in range(10):
            pixel_y = y * (src.shape[0] / 10)
            current_row.append([pixel_x, pixel_y])

            if shutter_duration:
                y_timestamp = current + shutter_duration * (pixel_y - src.shape[0] / 2)
            else:
                y_timestamp = current

            transform = getAccumulatedRotation(src.shape[1], src.shape[0], theta_x, theta_y, theta_z, timestamps, prev,
                                               current, f, gyro_delay, gyro_drift)

            output = cv2.perspectiveTransform(np.array([[pixel_x, pixel_y]], dtype="float32"), transform)
            current_row_transformed.append(output)

        pts.append(current_row)
        pts_transformed.append(current_row_transformed)

    o = utilities.meshwarp(src, pts_transformed)
    return o

I get the following error when it gets to output = cv2.perspectiveTransform(np.array([[pixel_x, pixel_y]], dtype="float32"), transform):

cv2.error: /Users/travis/build/skvark/opencv-python/opencv/modules/core/src/matmul.cpp:2271: error: (-215) scn + 1 == m.cols in function perspectiveTransform

Any help or suggestions would really be appreciated.

Answer 1

This implementation really needs to be changed in a future version, or the docs should be more clear.

From the OpenCV docs for perspectiveTransform():

src – input two-channel (...) floating-point array

Slant emphasis added by me.

>>> A = np.array([[0, 0]], dtype=np.float32)
>>> A.shape
(1, 2)

So we see from here that A is just a single-channel matrix, that is, two-dimensional. One row, two cols. You instead need a two-channel image, i.e., a three-dimensional matrix where the length of the third dimension is 2 or 3 depending on if you're sending in 2D or 3D points.

Long story short, you need to add one more set of brackets to make the set of points you're sending in three-dimensional, where the x values are in the first channel, and the y values are in the second channel.

>>> A = np.array([[[0, 0]]], dtype=np.float32)
>>> A.shape
(1, 1, 2)

Also, as suggested in the comments:

If you have an array points of shape (n_points, dimension) (i.e. dimension is 2 or 3), a nice way to re-format it for this use-case is points[np.newaxis]

It's not intuitive, and though it's documented, it's not very explicit on that point. That's all you need. I've answered an identical question before, but for the cv2.transform() function.