Generate point cloud from depth image

Question

I am trying to convert a depth image (RGBD) into a 3d point cloud. The solution I am currently using is taken from this post where:

• cx = image center height
• cy = image center width
• fx and fy = 250, chosen by iterating through a few options

The depth measurements have been taken from a pin hole camera and the point cloud is projecting away from the centre (example images below). Can anyone help me understand why and how I can solve this?

Answer 1

You may easily solve this using open3d package. Install it using sudo pip install -U open3d-python (not just open3d -- that's another package).

Once installed:

import open3d as o3d

rgbd = o3d.geometry.RGBDImage.create_from_color_and_depth(color, depth, convert_rgb_to_intensity = False)
pcd = o3d.geometry.PointCloud.create_from_rgbd_image(rgbd, pinhole_camera_intrinsic)

# flip the orientation, so it looks upright, not upside-down
pcd.transform([[1,0,0,0],[0,-1,0,0],[0,0,-1,0],[0,0,0,1]])

draw_geometries([pcd])    # visualize the point cloud

The above code assumes you have your color image in color and depth image in depth, check the samples coming with open3d for more information.

If you have your own camera intrinsic, you may replace pinhole_camera_intrinsic with those, but for the test run, the pinhole camera works more or less fine.