Find location of image inside bigger image

Question

So i have an image lets say small.png and a bigger image big.png .The small image occurs 2 times in the bigger image and i wish to find its location.

I tried using numpy and Image but got error

'JpegImageFile' object has no attribute '__getitem__'.I had png format earlier and it gave the same error.

Is there any other module or way to get this done.I am open to any .

The code as of now which throws error is

import numpy
from PIL import Image
here = Image.open(r"/Users/vks/Desktop/heren.jpg")
big = Image.open(r"/Users/vks/Desktop/Settingsn.jpg")
print here,big
herear = numpy.asarray(here)
bigar = numpy.asarray(big)
herearx = herear.shape[0]
hereary = herear.shape[1]

bigarx = bigar.shape[0]
bigary = bigar.shape[1]

print herearx , hereary , bigarx, bigary

stopx = bigarx - herearx + 1
stopy = bigary - hereary + 1

for x in range(0, stopx):
    for y in range(0, stopy):
        x2 = x+ herearx
        y2 = y + hereary
        pic = big[y:y2, x:x2] ===> error thrown here
        test = pic = here
        if test.all():
            print x,y
else:
    print "None"

There was another cv2 module but it just doesnt get installed on my mac. pip install cv2 fails saying on package found.

Answer 1

The following code works for me:

import numpy
from PIL import Image

here = Image.open(r"eye.png")
big  = Image.open(r"lenna.png")

herear = numpy.asarray(here)
bigar  = numpy.asarray(big)

hereary, herearx = herear.shape[:2]
bigary,  bigarx  = bigar.shape[:2]

stopx = bigarx - herearx + 1
stopy = bigary - hereary + 1

for x in range(0, stopx):
    for y in range(0, stopy):
        x2 = x + herearx
        y2 = y + hereary
        pic = bigar[y:y2, x:x2]
        test = (pic == herear)
        if test.all():
            print(x,y)

Output: 312 237

Graphically:

Test Images Used

lenna.png

eye.png

Note: It's important that you use a lossless image format when you create the smaller, cropped version (PNG is lossless, JPEG is most usually lossy). If you use a lossy format, you risk the pixel values being close, but not identical. The above code based off yours will only find exact pixel-by-pixel matches. The OpenCV template matching functions are a bit more flexible in this regard. That is not to say you couldn't modify your code to work just as well, you could. But as it stands, the code in this answer has that limitation.

More general version

Here, as a function, this gathers all matching coordinates and returns them as a list of (x,y) tuples:

import numpy as np
from PIL import Image

im_haystack = Image.open(r"lenna.png")
im_needle   = Image.open(r"eye.png")

def find_matches(haystack, needle):
    arr_h = np.asarray(haystack)
    arr_n = np.asarray(needle)

    y_h, x_h = arr_h.shape[:2]
    y_n, x_n = arr_n.shape[:2]

    xstop = x_h - x_n + 1
    ystop = y_h - y_n + 1

    matches = []
    for xmin in range(0, xstop):
        for ymin in range(0, ystop):
            xmax = xmin + x_n
            ymax = ymin + y_n

            arr_s = arr_h[ymin:ymax, xmin:xmax]     # Extract subimage
            arr_t = (arr_s == arr_n)                # Create test matrix
            if arr_t.all():                         # Only consider exact matches
                matches.append((xmin,ymin))

    return matches

print(find_matches(im_haystack, im_needle))

Update:

Given the images you provided, you'll notice that the way the matching is set up, it will only match one of the two here's. The top-left one is one the one you cropped for the needle image, so it matches exactly. The bottom-right image would need to match exactly pixel-for-pixel. With this implementation, even a single bit difference in one of the color channels would cause it to be ignored.

As it turns out, the two here's vary quite a bit more:

Tested Versions:

• Python 2.7.10, Numpy 1.11.1, PIL (Pillow) 1.1.7
• Python 3.5.0, Numpy 1.10.4, PIL (Pillow) 1.1.7

Answer 2

Here is the code using OpenCV that I research and found online.

import datetime

import numpy as np
from PIL import Image
import cv2

# sourceImagePath = ".\\COCImages\\ErrorPrompts\\AnyoneThere.png"
# needleImagePath = ".\\COCImages\\ErrorPrompts\\AnyoneThere_Search.png"
sourceImagePath = ".\\COCImages\\ErrorPrompts\\main.png"
needleImagePath = ".\\COCImages\\ErrorPrompts\\eye.png"


def start_findNeedle(sourceImagePath, needleImagePath):
    startTime = datetime.datetime.now()
    image = cv2.imread(sourceImagePath)
    # cv2.imshow('Rainforest', image)
    # cv2.waitKey(0)
    gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

    template = cv2.imread(needleImagePath, 0)

    result = cv2.matchTemplate(gray, template, cv2.TM_SQDIFF)
    min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(result)
    print("min_val: ", min_val)
    print("min_loc: ", min_loc)

    height, width = template.shape[:2]

    top_left = max_loc
    bottom_right = (top_left[0] + width, top_left[1] + height)
    cv2.rectangle(image, top_left, bottom_right, (0, 0, 255), 5)

    # cv2.imshow('Rainforest', image)
    # cv2.waitKey(0)
    # cv2.destroyAllWindows()

    endTime = datetime.datetime.now()
    totalTimeNeeded = endTime - startTime
    print("totalTimeNeeded: ", totalTimeNeeded)

    return min_loc

start_findNeedle(sourceImagePath, needleImagePath)

It runs faster than the previous code provide by @jedwards: Main Image (click here), Eye Image (click here)

Old version consumes 0:00:01.499806

My version consumes 0:00:00.009005

Note: there will a little warning shown which is libpng warning: iCCP: known incorrect sRGB profile because the image is not comes with sRGB format.

Hope this can solve and improve the time efficiency.

ADD ON : You may test with these two images too to check out the time efficiency

Main Image Main Image

Eye Image Eye Image

Update:

1. changed from cv2.TM_CCOEFF to cv2.TM_SQDIFF, which is more accurate.