Obstacle avoidance with stereo vision

Question

I'm working on a stereo-camera based obstacle avoidance system for a mobile robot. It'll be used indoors, so I'm working off the assumption that the ground plane is flat. We also get to design our own environment, so I can avoid specific types of obstacle that generate false positives or negatives.

I've already found plenty of resources for calibrating the cameras and getting the images lined up, as well as information on generating a disparity map/depth map. What I'm struggling with is techniques for detecting obstacles from this. A technique that instead worked by detecting the ground plane would be just as useful.

I'm working with openCV, and using the book Learning OpenCV as a reference.

Thanks, all

Answer 1

From the literature I've read, there are three main approaches:

1. Ground plane approaches determine the ground plane from the stereo data and assume that all points that are not on the plane are obstacles. If you assume that the ground is the dominant plane in the image, then you may be able to find it simply a plane to the reconstructed point cloud using a robust model-fitting algorithm (such as RANSAC).

2. Disparity map approaches skip converting the stereo output to a point cloud. The most popular algorithms I've seen are called v-disparity and uv-disparity. Both look for the same attributes in the disparity map, but uv-disparity can detect some types of obstacles that v-disparity alone cannot.

3. Point cloud approaches project the disparity map into a three-dimensional point cloud and process those points. One example is "inverted cone algorithm" that uses a minimum obstacle height, maximum obstacle height, and maximum ground inclination to detect obstacles on arbitrary, non-flat, terrain.

Of these three approaches, detecting the ground-plane is the simplest and least reliable. If your environment has sparse obstacles and a textured ground, it should be sufficient. I don't have much experience with disparity-map approaches, but the results look very promising. Finally, the Manduchi algorithm works extremely well under the widest range of conditions, including on uneven terrain. Unfortunately, it is very difficult to implement and is extremely computationally expensive.

References:

• v-Disparity: Labayrade, R. and Aubert, D. and Tarel, J.P.
Real time obstacle detection in stereovision on non flat road geometry through v-disparity representation
• uv-Disparity: Hu, Z. and Uchimura, K.
UV-disparity: an efficient algorithm for stereovision based scene analysis
• Inverted Cone Algorithm: Manduchi, R. and Castano, A. and Talukder, A. and Matthies, L.
Obstacle detection and terrain classification for autonomous off-road navigation

There are a few papers on ground-plane obstacle detection algorithms, but I don't know of a good one off the top of my head. If you just need a starting point, you can read about my implementation for a recent project in Section 4.2.3 and Section 4.3.4 of this design report. There was not enough space to discuss the full implementation, but it does address some of the problems you might encounter.