Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

pupil detection in OpenCV & Python

Tags:

python

opencv

I am doing pupil detection for my school project. It's my first time working with OpenCV and Python, using Python version 3.4.2 and OpenCV 3.1.0.

I am using the Raspberry Pi NoIR camera, and I am getting good images.

But i can't detect a pupil nicely (because of glint, lashes and shadows. I refer to some code on the web and the following is part of that code.

...

kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (5, 5))

# capture frames from the camera
for frame in camera.capture_continuous(rawCapture, format="bgr", use_video_port=True):

    image = frame.array
    cv2.imshow("image", image)


    gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
    retval, thresholded = cv2.threshold(gray, 80, 255, 0)
    cv2.imshow("threshold", thresholded)

    closed = cv2.erode(cv2.dilate(thresholded, kernel, iterations=1), kernel, iterations=1)
    #closed = cv2.morphologyEx(close, cv2.MORPH_CLOSE, kernel)

    cv2.imshow("closed", closed)

    thresholded, contours, hierarchy = cv2.findContours(closed, cv2.RETR_LIST, cv2.CHAIN_APPROX_NONE)

    drawing = np.copy(image)
    cv2.drawContours(drawing, contours, -1, (255, 0, 0), 2)

    for contour in contours:

        area = cv2.contourArea(contour)
        bounding_box = cv2.boundingRect(contour)

        extend = area / (bounding_box[2] * bounding_box[3])

        # reject the contours with big extend
        if extend > 0.8:
            continue

        # calculate countour center and draw a dot there
        m = cv2.moments(contour)
        if m['m00'] != 0:
            center = (int(m['m10'] / m['m00']), int(m['m01'] / m['m00']))
            cv2.circle(drawing, center, 3, (0, 255, 0), -1)

        # fit an ellipse around the contour and draw it into the image
        try:
            ellipse = cv2.fitEllipse(contour)
            cv2.ellipse(drawing, box=ellipse, color=(0, 255, 0))
        except:
            pass

    # show the frame
    cv2.imshow("Drawing", drawing)

    ...

Input image:

enter image description here

Output image:

enter image description here

How can I remove the parts of the image that are not related to the pupil, as shown above?

In addition to answers, any hints are also welcome.

like image 582
Yeon_ Avatar asked Apr 30 '16 14:04

Yeon_


People also ask

Can OpenCV be used for object detection?

OpenCV has a bunch of pre-trained classifiers that can be used to identify objects such as trees, number plates, faces, eyes, etc. We can use any of these classifiers to detect the object as per our need.

How does object detection work in OpenCV?

Basically, the Haar cascade technique is an approach based on machine learning where we use a lot of positive and negative images to train the classifier to classify between the images. Haar cascade classifiers are considered as the effective way to do object detection with the OpenCV library.


2 Answers

There are several things you can do. How well they work depends on how much variation there is in the images you want to apply the algorithm on. You could make several assumptions and then discard all the candidates that do not meet them.

remove small detections

At first I would consider removing candidates that are too small by adding this line at the beginning of your loop:

if area < 100:
    continue

The threshold was chosen randomly and worked well for this particular image. It removed almost all the false detection. Only the biggest one remains. But you have to check it against your other images and adapt it to your needs.

enter image description here

remove detections that are not round

Another assumption you can make is that pupils are usually round and you can remove every detection that is not 'round' enough. One simple measure of roundness is to look at the ratio of circumference to area.

circumference = cv2.arcLength(contour,True)
circularity = circumference ** 2 / (4*math.pi*area)

The circularity is about 2.72 for the shadow in the right side and 1.31 for the pupil.

improving roundness

You notice, that the contour of your pupil is not perfectly round because of reflections. You can improve this by computing the convex hull of the contours.

contour = cv2.convexHull(contour)

If you do that before computing the area and circumference you get circularity values of 1.01 and 1.37. (A perfect circle has a circularity of 1) This means the defect from the reflection was almost perfectly repaired. This might not be necessary in this case but could be useful in cases with more reflections.

enter image description here

like image 193
sietschie Avatar answered Oct 10 '22 18:10

sietschie


Some years ago I realized a very similar project. To everything added above I can suggest one little trick.

As you can see you have two reflections from any light source. One on the surface of the eye and second reflection on the surface of the glass.

If you will remove the glass (if it's possible) you will have a very bright reflection almost in the center of pupil. In that case you can ignore all objects without this bright reflection. This also will help you to find a position of eye in the space near camera

like image 38
VideoProcessingResearcher Avatar answered Oct 10 '22 18:10

VideoProcessingResearcher