Python and OpenCV - Improving my lane detection algorithm

Question

I'll need to detect road lane from a video. Here is my way of doing it.

1. Determine the Region of Interest (ROI) by slicing the image (focus the middle part)
2. Grayscale the ROI
3. Equalized the grayscaled ROI with cv2.equalizeHist
4. Apply Gaussian blur to (3)
5. Threshold (4) by using cv2.adaptiveThreshold
6. Skeletonize (5) by using skimage.morphology.skeletonize
7. Apply the cv2.HoughLines on (6)

For the cv2.HoughLines, I set up so that:

1. If rho is positive (which means the straight line is sloping to the right (bottom-up), it will only draw the line if it is at certain angles (I set up the range of the angle))
2. If rho is negative (straight line is sloping to the left (bottom-up), it will only draw the line if it is at certain angles)

This is my code for drawing the lines:

lines = cv2.HoughLines(image_bin, 1, np.pi/180, 50)
    try:
        range = lines.shape[0]
    except AttributeError:
        range = 0

    for i in xrange(range):
        for rho, theta in lines[i]:
            if rho > 0 and (np.pi*1/10 < theta < np.pi*4/10):
                a = np.cos(theta)
                b = np.sin(theta)
                x0 = a * rho
                y0 = b * rho
                x1 = int(x0 + 1000 * (-b))
                y1 = int(y0 + 1000 * (a))
                x2 = int(x0 - 1000 * (-b))
                y2 = int(y0 - 1000 * (a))

                cv2.line(roi, (x1, y1), (x2, y2), (0, 255, 0))

            if rho < 0 and (np.pi*7/10 < theta < np.pi*9/10):
                a = np.cos(theta)
                b = np.sin(theta)
                x0 = a * rho
                y0 = b * rho
                x1 = int(x0 + 1000 * (-b))
                y1 = int(y0 + 1000 * (a))
                x2 = int(x0 - 1000 * (-b))
                y2 = int(y0 - 1000 * (a))

                cv2.line(roi, (x1, y1), (x2, y2), (0, 255, 0))

If I didn't do what I just did above for the cv2.HoughLines function, I believe there will be a lot of unneeded lines drawn.

After adjusting the parameters and such, I got a quite good result but that is for just one picture. I don't think it will be that good for a video that will keep changing. What bother me the most is about my algorithm for drawing the needed lines (that is the road lane). Is there any better way? At least better than mine.

This is my result:

The original image:

Equalized histogram, thresholded, and skeletonized image of the ROI:

Final result:

Answer 1

I would recommend considering using the Probabilistic Hough Line Transform for your application. In OpenCV's Python API, it's implemented in the function, cv2.HoughLinesP. That will actually give you line segments so you won't need to calculate the endpoints. It's also much faster than the standard Hough Line Transform.

There are a few trade-offs though. For example, you may need to add logic to stitch line segments together. On the other hand, I've found that that's not really a bad thing. One of my toy projects (a self-driving miniature bus), used this approach, and having separate line segments to stitch together made it easier to deal with curved roads in which the standard hough line transform won't give you any lines at all.

Hope that helps.

EDIT: Regarding the details of line segment "stitching", it depends on what you're looking to accomplish. If you just want to display the road, and you're satisfied with there being some gaps between line segments, you may not need to do any stitching - just display all the line segments. In my application, I needed to determine the curvature of the lane, so I did some stitching to build up a model of each road lane which would include the average slope of the lane - that was used as input to a module responsible for controlling a servo to move the wheels accordingly.

By "stitching" I don't mean anything particularly sophisticated, but I'm not aware of any particular OpenCV functions to accomplish it. I just needed a way of associated line segments that were part of the same lane. So I processed the line segments returned from HoughLinesP from the top of the image working downward, and used each line segment's slope and y-intercept to determine where the line segments would intersect.