Recognizing image within image in C#

Question

I'd like to find an image (needle) within an image (haystack).

To keep things simple I take two screenshots of my desktop. One full size (haystack) and a tiny one (needle). I then loop through the haystack image and try to find the needle image.

1. capture needle and haystack screenshot
2. loop through haystack, looking out for haystack[i] == first pixel of needle
3. [if 2. is true:] loop through the 2nd to last pixel of needle and compare it to haystack[i]

Expected result: the needle image is found at the correct location.

I already got it working for some coordinates/widths/heights (A).

But sometimes bits seem to be "off" and therefore no match is found (B).

What could I be doing wrong? Any suggestions are welcome. Thanks.

---

var needle_height = 25;
var needle_width = 25;
var haystack_height = 400;
var haystack_width = 500;

---

A. example input - match

var needle = screenshot(5, 3, needle_width, needle_height); 
var haystack = screenshot(0, 0, haystack_width, haystack_height);
var result = findmatch(haystack, needle);

B. example input - NO match

var needle = screenshot(5, 5, needle_width, needle_height); 
var haystack = screenshot(0, 0, haystack_width, haystack_height);
var result = findmatch(haystack, needle);

---

1. capture needle and haystack image

private int[] screenshot(int x, int y, int width, int height)
{
Bitmap bmp = new Bitmap(width, height, PixelFormat.Format32bppArgb);
Graphics.FromImage(bmp).CopyFromScreen(x, y, 0, 0, bmp.Size);

var bmd = bmp.LockBits(new Rectangle(0, 0, bmp.Width, bmp.Height), 
  ImageLockMode.ReadOnly, bmp.PixelFormat);
var ptr = bmd.Scan0;

var bytes = bmd.Stride * bmp.Height / 4;
var result = new int[bytes];

Marshal.Copy(ptr, result, 0, bytes);
bmp.UnlockBits(bmd);

return result;
}

2. try to find a match

public Point findmatch(int[] haystack, int[] needle)
{
var firstpixel = needle[0];

for (int i = 0; i < haystack.Length; i++)
{
    if (haystack[i] == firstpixel)
    {
    var y = i / haystack_height;
    var x = i % haystack_width;

    var matched = checkmatch(haystack, needle, x, y);
    if (matched)
        return (new Point(x,y));
    }
}    
return new Point();
}

3. verify full match

public bool checkmatch(int[] haystack, int[] needle, int startx, int starty)
{
    for (int y = starty; y < starty + needle_height; y++)
    {
        for (int x = startx; x < startx + needle_width; x++)
        {
            int haystack_index = y * haystack_width + x;
            int needle_index = (y - starty) * needle_width + x - startx;
            if (haystack[haystack_index] != needle[needle_index])
                return false;
        }
    }
    return true;
}

Answer 1

Instead of making two screenshots of your desktop with a time interval between them, I would take a screenshot once and cut "needle" and "haystack" from those same bitmap source. Otherwise you have the risk of a change of your desktop contents between the two moments where the screenshots are taken.

EDIT: And when your problem still occurs after that, I would try to save the image to a file and try again with that file using your debugger, giving you a reproducible situation.