Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Finding an Image Inside Another Image

I'm trying to build an application that solves a puzzle (trying to develop a graph algorithm), and I don't want to enter sample input by hand all the time.

Edit: I'm not trying to build a game. I'm trying to build an agent which plays the game "SpellSeeker"

Say I have an image (see attachment) on the screen with numbers in it, and I know the locations of the boxes, and I have the exact images for these numbers. What I want to do is simply tell which image (number) is on the corresponding box.

Numbers

So I guess I need to implement

bool isImageInsideImage(Bitmap numberImage,Bitmap Portion_Of_ScreenCap) or something like that.

What I've tried is (using AForge libraries)

public static bool Contains(this Bitmap template, Bitmap bmp)
{
    const Int32 divisor = 4;
    const Int32 epsilon = 10;

    ExhaustiveTemplateMatching etm = new ExhaustiveTemplateMatching(0.9f);

    TemplateMatch[] tm = etm.ProcessImage(
        new ResizeNearestNeighbor(template.Width / divisor, template.Height / divisor).Apply(template),
        new ResizeNearestNeighbor(bmp.Width / divisor, bmp.Height / divisor).Apply(bmp)
        );

    if (tm.Length == 1)
    {
        Rectangle tempRect = tm[0].Rectangle;

        if (Math.Abs(bmp.Width / divisor - tempRect.Width) < epsilon
            &&
            Math.Abs(bmp.Height / divisor - tempRect.Height) < epsilon)
        {
            return true;
        }
    }

    return false;
}

But it returns false when searching for a black dot in this image.

How can I implement this?

like image 531
marvin Avatar asked Dec 01 '12 15:12

marvin


People also ask

How do I search for a photo of another picture?

All you have to do is go to images.google.com, click the camera icon that appears in the search bar, and: Paste in the URL of an image that you have seen somewhere online, or. Manually upload an image from your computer that you have saved, or. Drag an image from another window.

What is it called when you see a picture within a picture?

The Droste effect (Dutch pronunciation: [ˈdrɔstə]), known in art as an example of mise en abyme, is the effect of a picture recursively appearing within itself, in a place where a similar picture would realistically be expected to appear.


1 Answers

I'm answering my question since I've found the solution:

this worked out for me:

System.Drawing.Bitmap sourceImage = (Bitmap)Bitmap.FromFile(@"C:\SavedBMPs\1.jpg");
            System.Drawing.Bitmap template = (Bitmap)Bitmap.FromFile(@"C:\SavedBMPs\2.jpg");
            // create template matching algorithm's instance
            // (set similarity threshold to 92.5%)

           ExhaustiveTemplateMatching tm = new ExhaustiveTemplateMatching(0.921f);
                // find all matchings with specified above similarity

                TemplateMatch[] matchings = tm.ProcessImage(sourceImage, template);
                // highlight found matchings

           BitmapData data = sourceImage.LockBits(
                new Rectangle(0, 0, sourceImage.Width, sourceImage.Height),
                ImageLockMode.ReadWrite, sourceImage.PixelFormat);
            foreach (TemplateMatch m in matchings)
            {

                    Drawing.Rectangle(data, m.Rectangle, Color.White);

                MessageBox.Show(m.Rectangle.Location.ToString());
                // do something else with matching
            }
            sourceImage.UnlockBits(data);

The only problem was it was finding all (58) boxes for said game. But changing the value 0.921f to 0.98 made it perfect, i.e. it finds only the specified number's image (template)

Edit: I actually have to enter different similarity thresholds for different pictures. I found the optimized values by trying, in the end I have a function like

float getSimilarityThreshold(int number)
like image 68
marvin Avatar answered Oct 02 '22 23:10

marvin