How to segment bent rod for angle calculations?

Question

I'm trying to use OpenCV to segment a bent rod from it's background then find the bends in it and calculate the angle between each bend.

The first part luckily is trivial with a enough contrast between the foreground and background. A bit of erosion/dilation takes care of reflections/highlights when segmenting.

The second part is where I'm not sure how to approach it.

I can easily retrieve a contour (top and bottom are very similar so either would do), but I can't seem to figure out is how to get split the contour into the straight parts and the bend rods to calculate the angles.

So far I've tried simplyfying the contours, but either I get too many or too few points and it feels difficult to pin point the right settings to keep the straight parts straight and the bent parts simplified.

Here is my input image(bend.png)

And here's what I've tried so far:

#!/usr/bin/env python
import numpy as np
import cv2

threshold = 229
# erosion/dilation kernel
kernel = np.ones((5,5),np.uint8)
# contour simplification
epsilon = 0

# slider callbacks
def onThreshold(x):
    global threshold
    print "threshold = ",x
    threshold = x
def onEpsilon(x):
    global epsilon
    epsilon = x * 0.01
    print "epsilon = ",epsilon

# make a window to add sliders/preview to
cv2.namedWindow('processed')
#make some sliders
cv2.createTrackbar('threshold','processed',60,255,onThreshold)
cv2.createTrackbar('epsilon','processed',1,1000,onEpsilon)
# load image
img = cv2.imread('bend.png',0)
# continuously process for quick feedback
while 1:
    # exit on ESC key
    k = cv2.waitKey(1) & 0xFF
    if k == 27:
        break

    # Threshold
    ret,processed = cv2.threshold(img,threshold,255,0)
    # Invert
    processed = (255-processed)
    # Dilate
    processed = cv2.dilate(processed,kernel)
    processed = cv2.erode(processed,kernel)
    # Canny
    processed = cv2.Canny(processed,100,200)

    contours, hierarchy = cv2.findContours(processed,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)
    if len(contours) > 0:
        approx = cv2.approxPolyDP(contours[0],epsilon,True)
        # print len(approx)
        cv2.drawContours(processed, [approx], -1, (255,255,255), 3)
        demo = img.copy()
        cv2.drawContours(demo, [approx], -1, (192,0,0), 3)
    # show result
    cv2.imshow('processed ',processed)
    cv2.imshow('demo ',demo)


# exit
cv2.destroyAllWindows()

Here's what I've got so far, but I'm not convinced this is the best approach:

I've tried to figure this out visually and what I've aimed for is something along these lines:

Because the end goal is to calculate the angle between bent parts something like this feels simpler:

My assumption that fitting lines and compute the angles between pairs of intersecting lines could work:

I did a quick test using the HoughLines OpenCV Python tutorial, but regardless of the parameters passed I didn't get great results:

#!/usr/bin/env python
import numpy as np
import cv2

threshold = 229
minLineLength = 30
maxLineGap = 10
houghThresh = 15

# erosion/dilation kernel
kernel = np.ones((5,5),np.uint8)

# slider callbacks
def onMinLineLength(x):
    global minLineLength
    minLineLength = x
    print "minLineLength = ",x

def onMaxLineGap(x):
    global maxLineGap
    maxLineGap = x
    print "maxLineGap = ",x

def onHoughThresh(x):
    global houghThresh
    houghThresh = x
    print "houghThresh = ",x

# make a window to add sliders/preview to
cv2.namedWindow('processed')
#make some sliders
cv2.createTrackbar('minLineLength','processed',1,50,onMinLineLength)
cv2.createTrackbar('maxLineGap','processed',5,30,onMaxLineGap)
cv2.createTrackbar('houghThresh','processed',15,50,onHoughThresh)
# load image
img = cv2.imread('bend.png',0)
# continuously process for quick feedback
while 1:
    # exit on ESC key
    k = cv2.waitKey(1) & 0xFF
    if k == 27:
        break

    # Threshold
    ret,processed = cv2.threshold(img,threshold,255,0)
    # Invert
    processed = (255-processed)
    # Dilate
    processed = cv2.dilate(processed,kernel)
    processed = cv2.erode(processed,kernel)
    # Canny
    processed = cv2.Canny(processed,100,200)

    lineBottom = np.zeros(img.shape,np.uint8)

    contours, hierarchy = cv2.findContours(processed,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_SIMPLE)
    if len(contours) > 0:
        cv2.drawContours(lineBottom, contours, 0, (255,255,255), 1)

    # HoughLinesP
    houghResult = img.copy()
    lines = cv2.HoughLinesP(lineBottom,1,np.pi/180,houghThresh,minLineLength,maxLineGap)
    try:
        for x in range(0, len(lines)):
            for x1,y1,x2,y2 in lines[x]:
                cv2.line(houghResult,(x1,y1),(x2,y2),(0,255,0),2)
    except Exception as e:
        print e

    # show result
    cv2.imshow('lineBottom',lineBottom)
    cv2.imshow('houghResult ',houghResult)


# exit
cv2.destroyAllWindows()

Is this a feasible approach ? If so, what's the correct way of doing line fitting in OpenCV Python ?

Otherwise, that's the best way to tackle this problem ?

Update Following Miki's advise I've tried OpenCV 3's LSD and got nicer results than with HoughLinesP but it looks like there's still some tweaking needed, although it doesn't look other than cv2.createLineSegmentDetector there aren't many options to play with:

Answer 1

I know this is old but I found this after having a similar problem The method I used (after finding binary image) was along the lines of:

1. Find ends (points with fewest neighbors)
2. skeletonize (optional)
3. Starting at one end find a few closest points using skimage cdist
4. Perform a linear regression with these points and find all points in the image within a few pixels error of the line of best fit. I used query_ball_point
5. This gives additional points within the same straight line. Order them by distance from the last fiducial point. Some of these might be projections of the line onto distant parts of the object and should be deleted.
6. Repeat steps 4 and 5 until no more points are added.
7. Once no more points are added to the line you find the start of the next valid line by looking at R-squared for the fit. The line should have very high R squared eg. > 0.95 (depending on the image - I was getting > 0.99). Keep changing starting point until high R squared is achieved.
8. This gives a bunch of line segments from where it should be easy to find the angles between them. One potential problem occurs when the segment is vertical (or horizontal) and the slope becomes infinite. When this occurred I just flipped the axes around. You can also get around this by defining end points of a line and finding all points within a threshold distance from that line rather than doing the regression.

This involves a lot more coding than using the other methods suggested but execution time is fast and it gives much greater control over what is happening.