Need help understanding this rotation matrix math code

Question

private Mat4 calcLookAtMatrix(Vec3 cameraPt, Vec3 lookPt, Vec3 upPt) {
    Vec3 lookDir = Glm.normalize(Vec3.sub(lookPt, cameraPt));
    Vec3 upDir = Glm.normalize(upPt);

    Vec3 rightDir = Glm.normalize(Glm.cross(lookDir, upDir));
    Vec3 perpUpDir = Glm.cross(rightDir, lookDir);

    Mat4 rotMat = new Mat4(1.0f);
    rotMat.setColumn(0, new Vec4(rightDir, 0.0f));
    rotMat.setColumn(1, new Vec4(perpUpDir, 0.0f));
    rotMat.setColumn(2, new Vec4(Vec3.negate(lookDir), 0.0f));

    rotMat = Glm.transpose(rotMat);

    Mat4 transMat = new Mat4(1.0f);
    transMat.setColumn(3, new Vec4(Vec3.negate(cameraPt), 1.0f));

    return rotMat.mul(transMat);
}

This code generates a world to camera matrix that will be used with openGL for rendering.

I understand everything except how the rotation matrix is formed. Basically, it puts in normalized vectors and transposes the matrix, and somehow that comes out with a rotation matrix. Explain that math.

Answer 1

A transformation matrix transforms vertices from one space to another. Let's say that the initial space is space A and the final space is space B.

Both space A and space B are defined by a series of basis vectors. So space A has some basis vectors and an origin point. However, if you want to talk about basis vectors numerically, you have to talk about them relative to some other space. Otherwise, the numbers don't make sense, as a space defines what the numbers in a vector mean.

The transformation matrix that goes from space A to space B is the basis vectors for space A expressed relative to space B. The first column of the transformation is the direction of space A's X axis, relative to space B. The second column is A's Y axis and the third is A's Z axis.

What the code is doing is generating the basis vectors for camera space, relative to the given world space (as defined by the three vectors it is given). So it builds a matrix that takes vertices in camera space and spits them out in world space.

However, the matrix it wants to generate is the transform from world space to camera space. So it inverts the matrix before; the inverse of an A-to-B matrix is the B-to-A matrix. Fun fact: the inverse of a pure-rotation matrix is the same as its transpose, which is also much easier to compute.