Need to detect books via OpenCV Python

Question

See, I have been trying to detect books in a bookshelf:

I used Contours for bounding boxes. However, I just want to capture the actual book objects. If I lessen the threshold from Canny, it won't detect the book edges themselves but it detects the book titles or some images from the spine.

I used houghlines and it worked well for detecting the book edge. How can I apply bounding boxes but with houghlines instead of contours?

code I used for Contour finding:

    edges = cv2.Canny(blur,thresh,thresh*2)
    drawing = np.zeros(img.shape,np.uint8)  
    contours,hierarchy = cv2.findContours(edges,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)
    for cnt in contours:
        x,y,w,h = cv2.boundingRect(cnt)
        cv2.rectangle(img,(x,y),(x+w,y+h),(0,255,0),2)
        rect = cv2.minAreaRect(cnt)
        box = cv2.cv.BoxPoints(rect)
        box = np.int0(box)

where:

    img = cv2.imread('books3.jpg')
    gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
    blur = cv2.GaussianBlur(gray,(5,5),0)

For the houghlines:

    lines = cv2.HoughLines(edges,1,np.pi/180,120)
    for rho,theta in lines[0]:
        a = np.cos(theta)
        b = np.sin(theta)
        x0 = a*rho
        y0 = b*rho
        x1 = int(x0 + 1000*(-b))   
        y1 = int(y0 + 1000*(a))    
        x2 = int(x0 - 1000*(-b))   
        y2 = int(y0 - 1000*(a))

where:

    im = cv2.imread('books2.jpg')
    gray = cv2.cvtColor(im,cv2.COLOR_BGR2GRAY)
    edges = cv2.Canny(gray,100,300,apertureSize = 3)

Thank you so much in advance.

Answer 1

I am actually working on something similar myself. trying to segment the books from one another in a bookshelf. I wish to ask what is the progress on your side so far?

I have yet to tried contours method. However, what I did try was to pre-process the image, canny the image before using HoughLines. The image belows shows a rough result.

I admit I have to perfectly segment out the books either. As you can see in the image, there are more lines then I actually wanted due to the nature of the book spine. I am looking into preprocessing methods that can help me to rid such problem.

I noticed you mentioned that "If I lessen the threshold from Canny, it won't detect the book edges themselves but it detects the book titles or some images from the spine." Maybe for HoughLine Parameters, you can adjust the theta? for instance to 90 degrees so that the book titles, etc will not be detected.

You can also try HoughLineP which is basically Probabilistic Hough Line Transform. More details about that can be found in:

http://docs.opencv.org/doc/tutorials/imgproc/imgtrans/hough_lines/hough_lines.html

Hope my methods give some ideas.I also hope to hear updates from you in regards to your contour method. Hope we can share tips and work together as we have a common goal (: Hope to hear from you soon.