How can i pythonically us opencv to find a a basketball in an image?

Question

I've been laboring on a pet project for a bit on how to find a simple basketball in an image. I've tried a bunch of permutations of using hough.circles and transform , etc for the last few weeks but I cant seem to come anywhere close to isolating the basketball with the code examples and my own tinkering.

Here is an example photo: And here is the result after a simple version of circle finding code I've been tinkering with:

Anyone have any idea where I have gone wrong and how I can get it right?

Here is the the code I am fiddling with:

import cv2
import cv2.cv as cv # here
import numpy as np

def draw_circles(storage, output):
    circles = np.asarray(storage)
    for circle in circles:
        Radius, x, y = int(circle[0][3]), int(circle[0][0]), int(circle[0][4])
        cv.Circle(output, (x, y), 1, cv.CV_RGB(0, 255, 0), -1, 8, 0)
        cv.Circle(output, (x, y), Radius, cv.CV_RGB(255, 0, 0), 3, 8, 0)

 orig = cv.LoadImage('basket.jpg')
 processed = cv.LoadImage('basket.jpg',cv.CV_LOAD_IMAGE_GRAYSCALE)
 storage = cv.CreateMat(orig.width, 1, cv.CV_32FC3)
 #use canny, as HoughCircles seems to prefer ring like circles to filled ones.
 cv.Canny(processed, processed, 5, 70, 3)
 #smooth to reduce noise a bit more
 cv.Smooth(processed, processed, cv.CV_GAUSSIAN, 7, 7)

 cv.HoughCircles(processed, storage, cv.CV_HOUGH_GRADIENT, 2, 32.0, 30, 550)
 draw_circles(storage, orig)

 cv.imwrite('found_basketball.jpg',orig)

Answer 1

I agree with the other posters, that using the colour of the basketball is a good approach. Here is some simple code that does that:

import cv2
import numpy as np

im = cv2.imread('../media/basketball.jpg')

# convert to HSV space
im_hsv = cv2.cvtColor(im, cv2.COLOR_BGR2HSV)
# take only the orange, highly saturated, and bright parts
im_hsv = cv2.inRange(im_hsv, (7,180,180), (11,255,255))

# To show the detected orange parts:
im_orange = im.copy()
im_orange[im_hsv==0] = 0
# cv2.imshow('im_orange',im_orange)

# Perform opening to remove smaller elements
element = np.ones((5,5)).astype(np.uint8)
im_hsv = cv2.erode(im_hsv, element)
im_hsv = cv2.dilate(im_hsv, element)

points = np.dstack(np.where(im_hsv>0)).astype(np.float32)
# fit a bounding circle to the orange points
center, radius = cv2.minEnclosingCircle(points)
# draw this circle
cv2.circle(im, (int(center[1]), int(center[0])), int(radius), (255,0,0), thickness=3)

out = np.vstack([im_orange,im])
cv2.imwrite('out.png',out)

result:

I assume that:

1. Always one and only one basketball is present
2. The basketball is the principal orange item in the scene

With these assumptions, if we find anything the correct colour, we can assume its the ball and fit a circle to it. This way we don't do any circle detection at all.

As you can see in the upper image, there are some smaller orangey elements (from the shorts) which would mess up our ball radius estimate. The code uses an opening operation (erosion followed by dilation), to remove these. This works nicely for your example image. But for other images a different method might be better: using circle detection too, or contour shape, size, or if we are dealing with a video, we could track the ball position.

I ran this code (only modified for video) on a random short basketball video, and it worked surprisingly ok (not great.. but ok).

Answer 2

A few thoughts:

1. Filter by color first to simplify the image. If you're looking specifically for an orange basketball, you could eliminate a lot of other colors. I'd recommend using HSI color space instead of RGB, but in any case you should be able to exclude colors that are some distance in color 3-space from your trained basketball color.
2. Try substituting Sobel or some other kernel-based edge detector that doesn't rely on manual parameters. Display the edge image to see if it looks "right" to you.
3. Allow for weaker edges. In the grayscale image, the contrast between the basketball and the player's dark jersey is not as great as the difference between the white undershirt and the black jersey.
4. Hough may yield unexpected results if the object is only nominally circular in cross section, but is actually elongated or has noisy edges in the actual image. I usually write my own Hough algorithm and haven't touched the OpenCV implementation, so I'm not sure what parameter to change, but see if you can allow for fuzzier edges.
5. Maybe eliminate the smooth operation. In any case, try smooth before finding edges rather than the other way around.
6. Try to write your own rough Hough algorithm. Although a quickie implementation may not be as flexible as the OpenCV implementation, by getting your hands dirty you may stumble onto the source of the problem.