How to generate a point cloud?

Question

Using the setup above, I have the hough equations of the laser-lines on both sides of the plane. I have even filtered out the pixel points on the object. I also have extrinsic parameters of my camera taken from the checkerboard patterns I put on both sides of the plane (so I know {R1,T1} and {R2,T2})

Now I want to generate a point-cloud of the object. I see that I need to intersect the laser plane (Elaser) with viewing ray (r) from the camera to get a single 3D point (p), any clues on how can I do this?

---

UPDATE:

This is what I think I have to do, somebody may correct me if I am wrong

STEP1: Transform all points from (left plane) coordinate system to (right plane) coordinate system to get a single {R,T} from {R1, T1} and {R2, T2}

STEP2: Find the equation of laser plane using 2 laser points on left-plane and 1 laser point on right-plane using the same coordinate system (transformed to in STEP1)

STEP3: Find the 3D point on object using equation of laser plane and 2D point on object

---

UPDATE

This is what I have currently -

Answer 1

If I am not mistaken (I've never dealt with structured light), you need i) to recover the epipolar constraint between the camera and the laser, ii) determine the epipolar line for each point from the imaged laser stripe, iii) intersect this line with the known position of the laser stripe in laser coordinates, and iv) recover the 3D information by triangulation.

For best results, you should orientate the laser stripe perpendicular to the translation vector between the camera and the laser because this way the intersection in iii) above is well conditioned and is consequently most confidently recovered in presence of noise.