Converting OpenCV rotation and translation vectors to XYZ rotation and XYZ position

Question

There are Google results and stackoverflow posts that appear to answer this question, but the simple fact is I can't understand them. No matter how much I read, I can't get my head around quaternions and Euler angles and Rodriguez transforms and all that.

Anyone up for explaining to me like I'm 12 how to get from the rotation and translation vectors returned by OpenCV's solvePnP() method to xyz Position and xyz Rotation values that I can plug into a 3d graphics application?

This is for an augmented reality application in c++. I have the OpenCV part up and running, tracking a markerboard and sending rotation and translation vectors to the graphics program, but I have no idea how to use that information to pose my 3d object.

I really do want to understand the math behind this, and will appreciate any patient explanation of the theory here. But I'll also settle for a pointer to some code I can copy/paste and learn the theory another day. In fact I seem to learn stuff like this faster by seeing concrete code and working backward to the theory.

EDIT: Like there's this... which clearly should point me in the right direction, but it may as well be the plans for a time machine for all I know. It's occurred to me that I may be asking for remedial high school math, but it can't be the first time.

EDIT: Here's an example of the rotation vector and translation vector being returned from solvePnP()... converted into XML for the trip to the graphics application. Note that every one of these I've seen contains three rows and one column.

<Tvec type_id="opencv-matrix">
  <rows>3</rows>
  <cols>1</cols>
  <dt>f</dt>
  <data>
    -2.50094433e+01 -6.59909010e+00 1.07882790e+02
  </data>
</Tvec>
<Rvec type_id="opencv-matrix">
  <rows>3</rows>
  <cols>1</cols>
  <dt>f</dt>
  <data>
    -1.92100227e+00 -2.11300254e-01 2.80715879e-02
  </data>
</Rvec>

Answer 1

Initially I knew nothing about what formats OpenCV uses. So some research first. Looking at the documentation for solvePnP, it appears like its returning a matrix. (The Rodrigues2 function mentioned in an older version in the docs appears to be called Rodrigues these days.)

The description there indicates that you can expect a matrix with either a single row or a single column. The direction of that vector indicates the axis of rotation, while the length (or “norm”) of the vector gives the angle.

So your input is in angle-axis representation. Which you can interpret as a simultaneous rotation about the x, y and z axes. So if by “XYZ rotation” you mean simultaneous rotation, then you are already there. If, however, you mean some combination which rotates first about the x axis and then about the y axis and then about the z axis, you are in a strange setup, and probably should make sure you are using the correct API.

If it is Euler angles you are after, then calling them “XYZ rotation” is somewhat misleading. The Wikipedia article Rotation formalisms in three dimensions gives a list of the various formats, and how to convert between them. In particular, it does give formulas to convert from Axis and angle to matrix and from there to euler angles.

Make sure you are using the correct representation to match your graphics output machinery. Or if you have a choice, try to work directly on the rotation marix representation, as that should be easiest to understand and work with.

Answer 2

1. Extend your 3D vectors to homogeneous coordinates (4D). That means a 4D vector with components (X, Y, Z, 1)

2. Create a 4x4 transformation matrix based on your rotation and translation parameters. It is called an affine transformation. You are on the right track with the article about rotation matrices.

3. Multiply your coordinate vectors with the transformation matrix.

Check out this wikipedia section: http://en.wikipedia.org/wiki/Transformation_matrix#Affine_transformations

It has an example posed in 2d coordinates, I'm sure you get enough of the idea to extend to 3d.

Answer 3

My guess is that OpenCv already returns XYZ rotations. Take your left hand (left hand rule) and close your fist. Stick your thumb up (z axis), extend your index finger straight in front of you (x axis), then extend your middle finger to the right (y axis).

Any particular orientation vector can be given by up to three rotations. It is important to realize that you can get equivalent rotations. For example, rotate around the z-axis so that your index points to your right. An equivalent rotation would be Y,X, then Y again, without ever involving the Z axis.

However when you use the full rotation matrix (3x3), you don't have to bother with those conventions, because they are all merged together in the matrix. This is what the paper you linked is about, and not time travel, which is fortunate since most of math majors would be able to time travel, thus dooming mankind.

Rotation vectors however usually refers to around which vector in space your object must revolve.