Good algorithm to find similar areas in images?

Question

I want to search for similar areas in two images, but I don't know what works best. The areas are not scaled or transformed in any way, but may appear anywhere in both images (I want to know where). There is other stuff around them.

This is an example of what i want:

How can I do this?

Answer 1

1. segmentate image

   To obtain bound rectangles/polygon/mask of found areas

2. per each region compute

   • histogram
   • FFT or DCT and filter out unsignificant data (mostly high frequencies ... similar to JPEG comprimation)
   • size (width,heigth,area,perimeter length...)

3. find matches

   So compare each regions between images. Handle data from #2 as single dataset and compute the similarity between compared regions based on one from the following:

   • correlation coefficient
   • distance + tresholding
   • size coefficients (aspect ratio,perimeter/area,...)

4. for specific images you can create own custom comparison

   • for example here is mine for OCR
   • if you want the same size then you can easily add comparison of the sizes +/- some treshold

5. to improve precision

   You can divide each region to few subregions and compute #2 also for them to have more robust dataset but beware of the rotations.

   Also if you segmentation is based on color homogenity coefficient then you can also include that to the dataset

6. rotated images

   For that you need use features independend on rotation like:

   • histogram
   • color homogenity
   • use shapes for subregions invariant to rotation like co-centric circles ...

   Or find base feature/edge and rotate one image to match the other one position ...

7. polygons

   For polygon images you can vectorise image back to vector form and then use any polygon comparison algorithm