Getting median picture from sequence of images with OpenCV

Question

I have a sequence of images for which I want to calculate the median image (as to remove moving elements). Intuitively, hard-coding a loop to go through all the pixels would have a gross running time, as well as fairly large memory usage. Is there a way to easily do this in OpenCV? (I'm not interested in averaging, I need to do a median). I'm writing this for Android (using OpenCV4Android) so obviously computing power is limited.

Answer 1

As far as I know, there no OpenCV function that creates median image from sequence of images. I needed the same feature couple of years ago and I had to implement this myself. It is relatively slow because for each pixel you need to extract relevant pixel from multiple images (inefficient memory access) and calculate median (also a time consuming process).

Possible ways to increase efficiency are:

• There no need to compute median from all images. Small subset of images will be enough.
• You can find more efficient algorithms for finding median of some small groups. For example I used algorithm that can efficiently find median in group of nine values.

Answer 2

If the mean is ok:

Mat result(CV_64FC3, listImages[0].size());
for(int i = 0; i < listImages.size(); i++) {
    result += listImages[i];
}
result /= listImages.size();
result.convertTo(result, CV_8UC3);

EDIT:

This quick pseudo-median should make the trick:

// Following algorithm will retain the pixel which is the closest to the mean
// Computing Mean
Mat tmpResult = Mat.zeros(listImages[0].size(), CV_64FC3);
for(int i = 0; i < listImages.size(); i++) {
    tmpResult += listImages[i];
}
tmpResult /= listImages.size();
tmpResult.convertTo(tmpResult, CV_8UC3);
// We will now, for each pixel retain the closest to the mean
// Initializing result with the first image
Mat result(listImages[0].clone());
Mat diff1, diff2, minDiff;
for(int i = 1; i < listImages.size(); i++) {
    // Computing diff between mean/newImage and mean/lastResult
    absdiff(tmpResult, listImages[i], diff1);
    absdiff(tmpResult, result, diff2);
    // If a pixel of the new image is closer to the mean, it replaces the old one
    min(diff1, diff2, minDiff);
    // Get the old pixels that are still ok
    result = result & ~(minDiff - diff2);
    // Get the new pixels
    result += listImages[i] & (minDiff - diff2);
}

However the classic one should be also pretty fast. It is O(nb^2 * w * h) where nb is the number of images and w, h their width, height. The above is O(nb * w * h) with more operations on Mats.

The code for the classical one (almost all computations will be made in native):

Mat tmp;
// We will sorting pixels where the first mat will get the lowest pixels and the last one, the highest
for(int i = 0; i < listImages.size(); i++) {
    for(int j = i + 1; j < listImages.size(); j++) {
        listImages[i].copyTo(tmp);
        min(listImages[i], listImages[j], listImages[i]);
        max(listImages[j], tmp, listImages[j]);
    }
}
// We get the median
Mat result = listImages[listImages.size() / 2];