OpenCV estimate distance & normal vector from homography

Question

I'm matching a template from which I know my distance to & my normal vector to.

i.e. if my homography is the identity matrix then my camera is at Distance = 1.0m & my normal is at 0.

Now I have a second image in which I successfully aligned my template giving an homography:

     [0.82072, 0.05685, 66.75024]
 H = [0.02006, 0.86092, 39.34907]
     [0.00003, 0.00017, 01.00000]

I also have my camera matrix.

the opencv function :

cv::decomposeHomographyMat()

gives me 4 solutions for the Rotation(3x3 mat),Translation(3x1 mat) & Normal vector(3x1).

cv::warpPerspective()

Is able to map nearly perfectly the current view of the camera to my template.

So it should be possible to get the actual scaling (template to alignment) & the normal vector.

But I can't figure it out how to actually choose the correct solutions of cv::decomposeHomographyMat(), I'm I missing something?

EDIT: Posted "question" without the question...

Answer 1

I figured it out.

Step one:

I create a set of point in the ROI I can map to my template (points in the area defined by the corners of the ROI).

Step two:

Warp the points in ROI (from step one; 8 points are enough in all my tests & use case) with all the solutions of cv::decomposeHomographyMat()

Exclude all solutions that give a point3D(x, y, z) with a z value < 0 (i.e. point is behind the camera).

Step three:

At this point you should have one to two solutions left. All rotations matrixes should be the same, only the normal & translation matrix should differ.

Translations matrixes should verify:

Translation_Solution1 = -1* Translation_Solution2

Then compare your ROI area to you template area. If you ROI area is smaller than your template, it means that you template as been "scaled down", i.e. your camera did a translation on z in the negative values. Else you camera did a translation on the positive z values.

Chose the appropriate solution.

My error was to think that warpPerspective() was actually solving the Homography decomposition, but its not.

Answer 2

in paper Faugeras O D, Lustman F. Motion and structure from motion in a piecewise planar environment.1988 page 9 https://www.researchgate.net/publication/243764888_Motion_and_Structure_from_Motion_in_a_Piecewise_Planar_Environment