Rotation matrix that minimizes distance

Question

Let's say I have two points in 3D space (a and b) and a fixed axis/unit vector called n.

I want to create a rotation matrix that minimizes the euclidan distance between point a (unrotated) and the rotated point b.

E.g:

 Q := matrix_from_axis_and_angle (n, alpha);

 find the unknown alpha that minimizes sqrt(|a - b*Q|)

Btw - If a solution/algorithm can be easier expressed with unit-quaternions go ahead and use them. I just used matrices to formulate my question because they're more widely used.

---

Oh - I know there are some degenerated cases ( a or b lying exactly in line with n ect.) These can be ignored. I'm just looking for the case where a single solution can be calculated.

Answer 1

sounds fairly easy. Assume unit vector n implies rotation around a line parallel to n through point x0. If x0 != the origin, translate the coordinate system by -x0 to get points a' and b' relative to new coordinate system origin 0, and use those 2 points instead of a and b.

1) calculate vector ry = n x a

2) calculate unit vector uy = unit vector in direction ry

3) calculate unit vector ux = uy x n

You now have a triplet of mutually perpendicular unit vectors ux, uy, and n, which form a right-handed coordinate system. It can be shown that:

 a = dot(a,n) * n  +  dot(a,ux) * ux

This is because unit vector uy is parallel to ry which is perpendicular to both a and n. (from step 1)

4) Calculate components of b along unit vectors ux, uy. a's components are (ax,0) where ax = dot(a,ux). b's components are (bx,by) where bx = dot(b,ux), by = dot(b,uy). Because of the right-handed coordinate system, ax is always positive so you don't actually need to calculate it.

5) Calculate theta = atan2(by, bx).

Your rotation matrix is the one which rotates by angle -theta relative to coordinate system (ux,uy,n) around the n-axis.

This yields degenerate answers if a is parallel to n (steps 1 and 2) or if b is parallel to n (steps 4, 5).