Access pixel values within a contour boundary using OpenCV in Python

Question

I'm using OpenCV 3.0.0 on Python 2.7.9. I'm trying to track an object in a video with a still background, and estimate some of its properties. Since there can be multiple moving objects in an image, I want to be able to differentiate between them and track them individually throughout the remaining frames of the video.

One way I thought I could do that was by converting the image to binary, getting the contours of the blobs (tracked object, in this case) and get the coordinates of the object boundary. Then I can go to these boundary coordinates in the grayscale image, get the pixel intensities surrounded by that boundary, and track this color gradient/pixel intensities in the other frames. This way, I could keep two objects separate from each other, so they won't be considered as new objects in the next frame.

I have the contour boundary coordinates, but I don't know how to retrieve the pixel intensities within that boundary. Could someone please help me with that?

Thanks!

Answer 1

Going with our comments, what you can do is create a list of numpy arrays, where each element is the intensities that describe the interior of the contour of each object. Specifically, for each contour, create a binary mask that fills in the interior of the contour, find the (x,y) coordinates of the filled in object, then index into your image and grab the intensities.

I don't know exactly how you set up your code, but let's assume you have an image that's grayscale called img. You may need to convert the image to grayscale because cv2.findContours works on grayscale images. With this, call cv2.findContours normally:

import cv2
import numpy as np

#... Put your other code here....
#....

# Call if necessary
#img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

# Call cv2.findContours
contours,_ = cv2.findContours(img, cv2.RETR_LIST, cv2.cv.CV_CHAIN_APPROX_NONE)

contours is now a list of 3D numpy arrays where each is of size N x 1 x 2 where N is the total number of contour points for each object.

As such, you can create our list like so:

# Initialize empty list
lst_intensities = []

# For each list of contour points...
for i in range(len(contours)):
    # Create a mask image that contains the contour filled in
    cimg = np.zeros_like(img)
    cv2.drawContours(cimg, contours, i, color=255, thickness=-1)

    # Access the image pixels and create a 1D numpy array then add to list
    pts = np.where(cimg == 255)
    lst_intensities.append(img[pts[0], pts[1]])

For each contour, we create a blank image then draw the filled-in contour in this blank image. You can fill in the area that the contour occupies by specifying the thickness parameter to be -1. I set the interior of the contour to 255. After, we use numpy.where to find all row and column locations in an array that match a certain condition. In our case, we want to find the values that are equal to 255. After, we use these points to index into our image to grab the pixel intensities that are interior to the contour.

lst_intensities contains that list of 1D numpy arrays where each element gives you the intensities that belong to the interior of the contour of each object. To access each array, simply do lst_intensities[i] where i is the contour you want to access.

Answer 2

Answer from @rayryeng is excellent!

One small thing from my implementation is: The np.where() returns a tuple, which contains an array of row indices and an array of column indices. So, pts[0] includes a list of row indices, which correspond to height of the image, pts[1] includes a list of column indices, which correspond to the width of the image. The img.shape returns (rows, cols, channels). So I think it should be img[pts[0], pts[1]] to slice the ndarray behind the img.