What is a good metric for feature vector comparison and how to normalize them before comparison?

Question

Background:

I am working on a bottom up approach to image segmentation where in I first over-segment the image into small-regions/super-pixels/super-voxels and then I want to iteratively merge adjoining over-segmented regions based on some criterion. One criterion I have been playing with is to measure how similar in appearance are the two regions. To quantify appearance of a region, I use several measures -- intensity statistics, texture features etc. I lump all the features I compute for a region into a long feature vector.

Question:

Given two adjacent over-segmented regions R1 and R2, let F1 and F2 be the corresponding feature vectors. My questions are the following:

-- What are good metrics to quantify the similarity between F1 and F2?

-- How best to normalize F1 and F2 before quantifying their similarity with a metric? (using any supervised approach to normalization is not feasible because i dont want my algorithm to be tied to one set of images)

Solution in my mind:

Similarity(R1, R2) = dot_product(F1 / norm(F1), F2 / norm(F2))

In words, I first normalize F1 and F2 to be unit vectors and then use the dot product between the two vectors as a similarity measure.

I wonder if there are better ways to normalize them and compare them with a metric. I would be glad if the community can point me to some references and write out reasons why something else is better than the similarity measure I am using.

Answer 1

State of the art image segmentation algorithms use Conditional Random Fields over Superpixels (IMO SLIC algorithm is the best option). This type of algorithms capture the relationship between adjacent superpixels at the same time they classify each superpixel (normally using SSVM).

For superpixel classifying you will normally collect a bag of features for each of them, such as SIFT descriptors, histograms, or whatever feature you think it might help.

There are many papers that describe this process, here you have some of them which I find interesting:

• Associative Hierarchical CRFs for Object Class Image Segmentation
• Class Segmentation and Object Localization with Superpixel Neighborhoods
• Figure-ground segmentation using a hierarchical conditional random fields

However, there are not many libraries or software for dealing with CRF. The best you can find out there is this blog entry.