back-projecting a 2D point to 3D Plucker line

Question

I'm trying to build a tracker (hand tracking personal project), and for this reason I need to back-project a 2d point to a 3d line, using Plucker coordinates. (like ray tracing)

As an input, I have the 2d coordinates of a point and the projection matrix.

The information on the web about plucker coordinates give an overview of why they helpful, but there's no paper describing analytically the above mentioned procedure. (they just mention that they back-project to a plucker line, without any further description)

Can somebody please pinpoint me to the right direction?

Answer 1

Apparently there's no magic behind this, I was looking for a formula/theorem resulting immediately from my 'inputs' to plucker coordinates, while there's no such thing.

As inputs, we have

• 2D coordinates of a (projected) point
• the projection matrix

Using these two inputs, we can back-project this 2d point to a ray (3D line). All 3D points of this 3D line are projected to the same 2D point. The ray by default passes through the camera center (or projection center,etc).

For necessary equations, see

• Hartley/Zisserman book @ page 162 @ equation 6.14, OR
• Equation 23 -> http://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/FUSIELLO4/tutorial.html

The general idea is that, in order to define a line, you need 2 points. We choose to find (with equations from the above mentioned sources)

• The camera center (all projection rays pass by default through this point)
• A point of the ray @ infinity (this is nice, because a point at infinity is also the direction vector of the line -> this is needed later-on for the plucker line representation)

(we could have found the camera center and another arbitrary point, but then we would need an extra step to find the direction of the line, by subtracting the coordinates of these two points.)

To sum up, we have found

• The camera center (p)
• The direction (d) of the line (point @ infinity) (Points at infinity are equivalent to directions)

These are enough to represent a line, but this representation is not optimal when we have to compute for example distances of 3D points to this line in our algorithm. This is why, after finding this representation (no magic to give us plucker lines immediately), we change our line-representation to plucker-line-representation

Plucker line is just another representation of the line that needs:

• The direction of the line (we already have it!!! -> d -> point at infinity)

• The 'moment' (m) of the line, which is easy to compute from the previous representation:

  m=p^d (^ -> cross product)

I hope this clears things for anyone that will need this in the future, I think it's a very easy thing, but in the beginning things might not be so apparent.

For a practical scenario, why one would use this plucker-line-representation, please check

• http://files.is.tue.mpg.de/dtzionas/Hand-Object-Capture/IJCV_Hand_Object_Capture.pdf
• https://pages.iai.uni-bonn.de/gall_juergen/download/jgall_tpami09.pdf