Detect uniform images that (most probably) are not photographs

Question

Take a look at these two example images:

I would like to be able to identify these types of images inside large set of photographs and similar images. By photograph I mean a photograph of people, a landscape, an animal etc.

I don't mind if some photographs are falsely identified as these uniform images but I wouldn't really want to "miss" some of these by identifying them as photographs.

The simplest thing that came to my mind was to analyze the images pixel by pixel to find highest and lowest R,G,B values (each channel separately). If the difference between lowest and highest value is large, then there are large color changes and such image is probably a photograph.

Other idea was to analyze the Hue value of each pixel in similar fashion. The problem is that in HSL model orangish-red and pinkish-red have roughly 350 degree difference when looking clockwise and 10 degree difference when looking counterclockwise. So I cant just compare each pixel's Hue component because I'll get some weird results.

Also, there is a problem of noise - one white or black pixel will ruin tests like that. So I would need to somehow exclude extreme values if there are only few pixels with such extremes. But at this point it gets more and more complicated and I'm feeling it's not the best approach.

I was also thinking about bumping contrast to the max and then running test like the RGB one I described above. It would probably make things easier but still one or two abnormal pixels would ruin the test anyway. How to deal with such cases?

I don't mind running few different algorithms that would cover different image types. But please note that I'm dealing with images from digital cameras so 6MP, 12MP or even 16MP are quite common. Because of that running computation intensive algorithms is not desired. I deal with hundreds or even thousands of images and have only limited CPU resources for image processing. Lets say a second or two per large image is max what I can accept.

I'm aware that for example a photograph of a blue sky might trigger a false positive, but that's OK. False positives are better than misses.

Answer 1

This how I would do it (Whole Method below, at the bottom of post, but just read from top to bottom):

Your quote:

"By photograph I mean a photograph of people, a landscape, an animal etc."

My response to your quote:

This means that such images have edges, contours. The images you are trying to separate out, no edges or little contours(for the second example image at least)

Your quote:

one white or black pixel will ruin tests like that. So I would need to somehow exclude extreme values if there are only few pixels with such extremes

My response:

Minimizing the noise through methods like DoG(Difference of Gaussian), etc will reduce the noisy, individual pixels

So I have taken your images and run it through the following codes:

cv::cvtColor(image, imagec, CV_BGR2GRAY); // where image is the example image you shown
cv::GaussianBlur(imagec,imagec, cv::Size(3,3), 0, 0, cv::BORDER_DEFAULT ); //blur image
cv::Canny(imagec, imagec, 20, 60, 3);

Results for example image 1 you gave:

As you can see after going through the code, the image became blank(totally black). The image quite big, hence bit difficult to show all in one window.

Results for example 2 you showed me:

The outline can be seen, but one method to solve this, is to introduce an ROI of about 20 to 30 pixels from the dimension of the image, so for instance, if image dimension is 640x320, the ROI may be 610x 290, where it is placed in the center of the image.

So now, let me introduce you my real method:

1) run all the images through the codes above to find edges

2) check which images doesn't have any any edges( images with no edges will have 0 pixel with values more then 0 or little pixels with values more then 0, so set a slightly higher threshold to play it safe? You adjust accordingly, how many pixels to identify your images )

3) Save/Name out all the images without edges, which will be the images you are trying to separate out from the rest.

4) The end.

EDIT(TO ANSWER THE COMMENT, would have commented back, but my reply is too long):

true about the blurring part. To minimise usage of blurring, you can first do an "elimination-like process", so those smooth like images in image 1 will be already separated and categorised into images you looking for.

From there you do a second test for the remaining images, which will be the "blurring".

If you really wish to avoid blurring, what I notice is that your example image 1 can be categorised as "smooth surface" while your example image 2 can be categorised as "rough-like surface", meaning which it be noisy, which led me to introduce the blurring in the first place.

From my experience and if I do remember correctly, such rough-like surfaces is very good in "watershed" or "clustering through colour" method, they blend very well, unlike the smooth images.

Since the leftover images are high chances of rough images, you can try the watershed method, and canny, you will find it be a black image, if I am not wrong. Try a line maybe like this:

pyrMeanShiftFiltering( image, images, 10, 20, 3)

I am not very sure if such method will be more expensive than Gaussian blurring. but you can try both and compare the computational speed for both.

In regard to your comment on grayscale images:

Converting to grayscale sounds risky - loosing color information may trigger lot's of false positives

My answer:

I don't really think so. If your images you are trying to segment out are of one colour, changing to grayscale doesn't matter. Of course if you snap a photo of a blue sky, it might cause to be a false negative, but as you said, those are ok.

If you think about it, images with people, etc inside, the intensity change differs quite a lot. (of course unless your photograph have extreme cases, like a green ball on a field of grass)

I do admit that converting to grayscale loses information. But in your case, I doubt it will affect much, in fact, working with grayscale images is faster and less expensive.

Answer 2

I would use entropy based approach. I don't have any custom code to share, but the following blog entry should push you in right direction.

http://envalo.com/image-cropping-php-using-entropy-explained/

The thing is, that the uniform images will have very low entropy compared to those with something interesting in them.

So the question is to find the correct threshold and process the whole set.

Answer 3

I would generate a color histogram for each image and compare how much they differ from a given pattern.

Maybe you want to normalize the brightness first to simplify the matching.