Which features does project tango use

Question

Does project tango extract any visual features per frame (such as ORB or SIFT/SURF)? or the entire point cloud is just 3D points extracted from depth camera. If so, is it possible to know which algorithm are they using? Is it just corners ?

I would like to dump 3D point cloud along with corresponding features and wondering if it is all possible in real-time.

Answer 1

Unfortunately, they don't expose which features they use. All you get is XYZ + Confidence. Here's the realtime point cloud callback, from the C API:

TangoErrorType TangoService_connectOnPointCloudAvailable(
    void(*)(void *context, const TangoPointCloud *cloud) TangoService_onPointCloudAvailable,
    ...
);

See:

• https://developers.google.com/tango/apis/c/reference/group/depth
• https://developers.google.com/tango/apis/java/reference/TangoPointCloudData
• https://developers.google.com/tango/apis/unity/reference/class/tango/tango-unity-depth

TangoPointCloud is defined here:

• https://developers.google.com/tango/apis/c/reference/struct/tango-point-cloud#struct_tango_point_cloud
• https://developers.google.com/tango/apis/java/reference/TangoPointCloudData
• https://developers.google.com/tango/apis/unity/reference/class/tango/tango-point-cloud-data

As an aside, if you regard Tango's objective as being a portable API that sits atop various different sensors and hardware platforms, then it makes sense that they wouldn't expose the details of the underlying depth estimation method. It might change, from one device to the next.

BTW, they also keep the internals of their ADF (Area Description File) format secret.