Image Histogram Comparison

Question

I was trying to do histogram image comparison between two RGB images which includes heads of the same persons and non-heads to see the correlation between them. The reason I am doing this is because after performing scanning using HOG to check whether the scanning window is a head or not, I am now trying to track the same head throughout consequent frames and also I want to remove some clear false positives.

I currently tried both RGB and HSV histogram comparison and used Euclidean Distance to check the difference between the histograms. The following is the code I wrote:

%RGB histogram comparison

 %clear all;
img1 = imread('testImages/samehead_1.png');
img2 = imread('testImages/samehead_2.png');

img1 = rgb2hsv(img1);
img2 = rgb2hsv(img2);

%% calculate number of bins = root(pixels);
[rows, cols] = size(img1);
no_of_pixels = rows * cols;
%no_of_bins = floor(sqrt(no_of_pixels));
no_of_bins = 256;

%% obtain Histogram for each colour
% -----1st Image---------
rHist1 = imhist(img1(:,:,1), no_of_bins);
gHist1 = imhist(img1(:,:,2), no_of_bins);
bHist1 = imhist(img1(:,:,3), no_of_bins);

hFig = figure;
hold on;
h(1) = stem(1:256, rHist1);
h(2) = stem(1:256 + 1/3, gHist1);
h(3) = stem(1:256 + 2/3, bHist1);

set(h, 'marker', 'none')
set(h(1), 'color', [1 0 0])
set(h(2), 'color', [0 1 0])
set(h(3), 'color', [0 0 1])

hold off;

% -----2nd Image---------
rHist2 = imhist(img2(:,:,1), no_of_bins);
gHist2 = imhist(img2(:,:,2), no_of_bins);
bHist2 = imhist(img2(:,:,3), no_of_bins);

hFig = figure;
hold on;
h(1) = stem(1:256, rHist2);
h(2) = stem(1:256 + 1/3, gHist2);
h(3) = stem(1:256 + 2/3, bHist2);

set(h, 'marker', 'none')
set(h(1), 'color', [1 0 0])
set(h(2), 'color', [0 1 0])
set(h(3), 'color', [0 0 1])

%% concatenate values of a histogram in 3D matrix
% -----1st Image---------
M1(:,1) = rHist1;
M1(:,2) = gHist1;
M1(:,3) = bHist1;

% -----2nd Image---------
M2(:,1) = rHist2;
M2(:,2) = gHist2;
M2(:,3) = bHist2;

%% normalise Histogram
% -----1st Image---------
M1 = M1./no_of_pixels;
% -----2nd Image---------
M2 = M2./no_of_pixels;

%% Calculate Euclidean distance between the two histograms
E_distance = sqrt(sum((M2-M1).^2));

The E_distance consists of an array containing 3 distances which refer to the red histogram difference, green and blue.

The Problem is:

• When I compare the histogram of a non-head(eg. a bag) with that of a head..there is a clear difference in the error. So this is acceptable and can help me to remove the false positive.
• However! When I am trying to check whether the two images are heads of the same person, this technique did not help at all as the head of another person gave a less Euclidean distance than that of the same person.

Can someone explain to me if I am doing this correctly, or maybe any guidance of what I should do?

PS: I got the idea of the LAB histogram comparison from this paper (Affinity Measures section): People Looking at each other

Answer 1

Color histogram similarity may be used as a good clue for tracking by detection, but don't count on it to disambiguate all possible matches between people-people and people-non-people.

According to your code, there is one thing you can do to improve the comparison: currently, you are working with per-channel histograms. This is not discriminative enough because you do not know when R-G-B components co-occur (e.g. you know how many times the red channel is in range 64-96 and how many times the blue is in range 32-64, but not when these occur simultaneously). To rectify this, you must work with 3D histograms, counting the co-occurrence of colors). For a discretization of 8 bins per channel, your histograms will have 8^3=512 bins.

Other suggestions for improvement:

1. Weighted assignment to neighboring bins according to interpolation weights. This eliminates the discontinuities introduced by bin quantization
2. Hierarchical splitting of detection window into cells (1 cell, 4 cells, 16 cells, etc), each with its own histogram, where the histograms of different levels and cells are concatenated. This allows catching local color details, like the color of a shirt, or even more local, a shirt pocket/sleeve.
3. Working with the Earth Mover's Distance (EMD) instead of the Euclidean metric for comparing histograms. This reduces color quantization effects (differences in histograms are weighted by color-space distance instead of equal weights), and allows for some error in the localization of cells within the detection window.
4. Use other cues for tracking. You'll be surprised how much the similarity between the HoG descriptors of your detections helps disambiguate matches!