Transformation Concept in OpenCV

Question

I am new to opencv. and I am right now going through with the concept of Image Transformation in OpenCV. So my question is,

1) Why does Affine Transformation use 2*3 matrix and perspective transformation use 3*3 matrix?

2) When to use Affine transformation and Perspective transformation over each other?

Any Suggestions?

Answer 1

1) It is not a question about OpenCV but rather about mathematics. Applying affine transformation to point (x,y) means the following:

x_new = a*x + b*y + c;
y_new = d*x + e*y + f;

And so affine transform has 6 degrees of freedom: a, b, c, d, e, f. They are stored in 2x3 matrix: a, b, c in the first row, and d, e, f in the second row. You can apply transform to a point by multiplying of matrix and vector.

Perspective transform of (x,y) would be:

z = g*x + h*y + 1;
x_new = (a*x + b*y + c)/z;
y_new = (d*x + e*y + f)/z;

As you can see it has 8 degrees of freedom that are stored in 3x3 matrix. Third row is g, h, 1.

See also homogeneous coordinates for more information about why this representation is so convenient.

2) Affine transformation is also called 'weak perspective' transformation: if you are looking at some scene from different perspective but size of the scene is small relatively to distance to the camera (i.e. parallel lines remain more or less parallel), than you may use affine transform. Otherwise perspective transform will be required.

Answer 2

It is better to consider a hole family of transformations - then you really remember what is what. Let’s go from simplest to complex ones: 1. Euclidean - this is a rigid rotation in plane plus translation. Basically all you can do with a piece of paper lying on the table. 2. Similarity - more general transformation where you can rotate, translate and also scale (hence it is non-rigid); 3. Affine - adds another operation - shear - which would make a parallelogram from a rectangle. This kind of sheer happens during orthographic projection or when objects are viewed from a long distance (compared to their size); parallel lines are still preserved. 4. Homography or perspective transformation - most general transformation and it will make a trapezoid out of rectangle (that is different amount of shear applied to each side). This happens when projecting planar objects from close distance. Remember how train trucks converge to a point at infinity? hence the name perspective. It also means that unlike other transformations we have to apply a division at some point. This what a third row does when we convert from Homogeneous to Cartesian coordinates we divide by a value in a last third row.

This transformation is the only one that cannot be optimally computed using linear algebra and requires non-linear optimization (coz of devision). In camera projections homography happens in three cases: 1. between flat surface and its image; 2. between arbitrary images of 3D scene when camera rotates but not translates; 3. during zoom operation. In other words whenever a flat camera sensor crosses the same optical rays you have a homography.