3D rendering in OpenCV

Question

I am doing a project on 3D rendering of a scene. I am using OpenCV. The steps I am doing are like this:

1. Taking two images of a scene.
2. Calculating object correspondence using SURF feature matching.
3. Calculating camera fundamental matrix.
4. Calculating the Disparity image.

Now I have two questions

1. After calculating fundamental matrix how can I calculate the Q matrix? (I can't calibrate the camera)

2. How can I render in 3D using opencv or any other library?

Answer 1

For the 3D part, you can render your scene with OpenGL or with PCL. You've two solutions:

• For each pixel, you make a point with the right color extracted from the camera's image. This will give you a point cloud which can be processed with PCL (for 3D features extraction for example).
• You apply a triangulation algorithm, but in order to apply this algorithm you must have the extrinsic matrices of your camera.

You can find more information about these techniques here:

• Point Cloud technique
• Triangulation algorithm

If you want to use OpenGL, you have to open a valid OpenGL context. I recommend you the SFML library or Qt. These libraries are very easy to use and have a good documentation. Both have tutorials about 3D rendering with OpenGL.

Answer 2

you can have Q matrix from stereo rectification via openCV method:

cv::stereoRectify

I think you want the Q matrix to reconstruct the 3D. However, you can reconstruct from intrinsic parameters via:

X = (u-cu)*base/d
Y = (v-cv)*base/d
Z = f*base/d 

where (u,v) is a 2D point in the image coordinate system, (cu,cv) is the principal point of the camera, f is the focal length, base is the baseline, d is the disparity and (X,Y,Z) is a 3D point in the camera coordinate system.