Calculate Camera parameters and HandEyeCalibration

Question

I'm trying to port an old Augmented Reality project, which is based on the metaio framework for an other see-through HMD.

Cause metaio is bought by apple, it's available documentation and support is very limited and this new HMD is not supported out of the box.

Everything is already working, besides the camera calibration / hand eye calibration.

I have already calculated a Projection matrix P (using SPAAM) which transforms points in the camera coordinate system to the screen (Tested by custom rendering). Unfortunately I can't set the projection matrix directly in metaio but have to separately set the camera parameters and Hand Eye calibration for rendering.

Therefore I have extracted the extrinsic and intrinsic camera parameters from the Projection matrix P as described here.

But those are not working correctly when set. The result seems to be not completely off, cause the rendered scale seems to be correct and manual measured distance between camera and eye is about 126mm (translation on x axis).

Projection matrix P:

Intrinsics (resolution: 1280 x 720):

Extrinsics (hec?):

I guess I'm missing some additional step I'm not aware of. I appreciate any ideas and help.

Additional information:

• metaio is using right handed coordinate systems with camera looking in negative z axsis, x to the right and y up
• metaio documentaion for hand eye calibration states: Transformation from camera to eye (display) coordinates
• Since The translation part of HEC on the x- axis has to be positive (camera is on the right side of the HMD) I assume it's transformation is specified from eye to camera?

Answer 1

Does Metaio automatically invert your matrices for you? Because if it doesn't, this could be your problem. As y=0 is at the top of the screen in computer graphics many of these image/camera processing apps come with an option to invert the transformation matrix for you. If it doesn't you may want to just try manually inverting your transformation matrix to see if that works instead.