How to find average intensity of OpenCV contour in realtime

Question

I have a image with about 50 to 100 small contours. I wish to find the average intensity[1] of each of these contours in real-time[2]. Some of the ways I could think of was

1. Draw contour with FILLED option for each contour; use each image as a mask over the original image, thus find the average. But I presume that this method won't be real-time at first glance.

2. Study OpenCV implementation of drawContour function with the FILLED option and access the pixels enclosed by the contour in the same manner. But the code seems really complex and not readily understandable.

3. Calculate the minimum area rectangle, find all the points inside the rectangle using transformation, and find average of points that are non-zero. Again, seems complex approach.

Is there an easier, efficient way to do this?

[1] Average of all pixel intensities enclosed by each of the non-overlapping contours

[2] About 25, (960 x 480) pixel images per sec on a 2.66 Ghz desktop PC

Answer 1

I was unable to come up with any methods substantially different than your suggested approaches. However, I was able to do some timings that may help guide your decision. All of my timings were run on a 1280*720 image on an iMac, limited to finding 100 contours. The timings will of course be different on your machine, but the relative timings should be informative.

For each test case, the following are declared:

std::vector<std::vector<cv::Point>> cont;  // Filled by cv::findContours()
cv::Mat labels = cv::Mat::zeros(image.size(), CV_8UC1);     
std::vector<float> cont_avgs(cont.size(), 0.f); // This contains the averages of each contour

Method 1: 19.0ms

Method 1 is conceptually the simplest, but it is also the slowest. Each contour is labeled by assigning a unique color to each contour. Values of each labelled component are summed by iterating through every pixel in the image.

for (size_t i = 0; i < cont.size(); ++i)
{
    // Labels starts at 1 because 0 means no contour
    cv::drawContours(labels, cont, i, cv::Scalar(i+1), CV_FILLED);
}

std::vector<float> counts(cont.size(), 0.f);
const int width = image.rows;
for (size_t i = 0; i < image.rows; ++i)
{
    for (size_t j = 0; j < image.cols; ++j)
    {
        uchar label = labels.data[i*width + j];

        if (label == 0)
        {
            continue;   // No contour
        }
        else
        {
            label -= 1; // Make labels zero-indexed
        }

        uchar value = image.data[i*width + j];
        cont_avgs[label] += value;
        ++counts[label];
    }
}
for (size_t i = 0; i < cont_avgs.size(); ++i)
{
    cont_avgs[i] /= counts[i];
}

Method 3: 15.7ms

An unmodified Method 3 has the simplest implementation and is also the fastest. All contours are filled to use as a mask for finding the mean. The bounding rectangle of each contour is computed, and then the mean is calculated using the mask within the bounding box.

Warning: This method will give incorrect results if any other contours are within the bounding rectangle of the contour of interest.

cv::drawContours(labels, cont, -1, cv::Scalar(255), CV_FILLED);

for (size_t i = 0; i < cont.size(); ++i)
{
    cv::Rect roi = cv::boundingRect(cont[i]);
    cv::Scalar mean = cv::mean(image(roi), labels(roi));
    cont_avgs[i] = mean[0];
}

Modified Method 3: 17.8ms

Making slight modifications to Method 3 increases execution time slightly, but gains the benefit of giving correct results regardless of contour positions. Each contour is individually labeled, and the mean is calculated using only the mask for that contour.

for (size_t i = 0; i < cont.size(); ++i)
{
    cv::drawContours(labels, cont, i, cv::Scalar(i), CV_FILLED);
    cv::Rect roi = cv::boundingRect(cont[i]);
    cv::Scalar mean = cv::mean(image(roi), labels(roi) == i);
    cont_avgs[i] = mean[0];
}