Simulate mouse click in MSPaint

Question

I have a console application which should paint a random picture in MSPaint (mouse down -> let the cursor randomly paint something -> mouse up. This is what I have so far (I added comments to the Main method for better understanding what I want to achieve):

[DllImport("user32.dll", CallingConvention = CallingConvention.StdCall)]
public static extern void mouse_event(long dwFlags, uint dx, uint dy, long cButtons, long dwExtraInfo);
private const int MOUSEEVENTF_LEFTDOWN = 0x201;
private const int MOUSEEVENTF_LEFTUP = 0x202;
private const uint MK_LBUTTON = 0x0001;

public delegate bool EnumWindowsProc(IntPtr hWnd, IntPtr parameter);

[DllImport("user32.dll", SetLastError = true)]
static extern IntPtr FindWindow(string lpClassName, string lpWindowName);

[DllImport("user32.dll", SetLastError = true)]
public static extern IntPtr FindWindowEx(IntPtr parentHandle, IntPtr childAfter, string className, string windowTitle);

[DllImport("user32.dll", CharSet = CharSet.Auto)]
static extern IntPtr SendMessage(IntPtr hWnd, UInt32 Msg, IntPtr wParam, IntPtr lParam);

[DllImport("user32.dll", SetLastError = true)]
public static extern bool EnumChildWindows(IntPtr hwndParent, EnumWindowsProc lpEnumFunc, IntPtr lParam);

static IntPtr childWindow;

private static bool EnumWindow(IntPtr handle, IntPtr pointer)
{
    childWindow = handle;
    return false;
}

public static void Main(string[] args)
{
    OpenPaint(); // Method that opens MSPaint
    IntPtr hwndMain = FindWindow("mspaint", null);
    IntPtr hwndView = FindWindowEx(hwndMain, IntPtr.Zero, "MSPaintView", null);
    // Getting the child windows of MSPaintView because it seems that the class name of the child isn't constant
    EnumChildWindows(hwndView, new EnumWindowsProc(EnumWindow), IntPtr.Zero);
    Random random = new Random();
    Thread.Sleep(500);

    // Simulate a left click without releasing it
    SendMessage(childWindow, MOUSEEVENTF_LEFTDOWN, new IntPtr(MK_LBUTTON), CreateLParam(random.Next(10, 930), random.Next(150, 880)));
    for (int counter = 0; counter < 50; counter++)
    {
        // Change the cursor position to a random point in the paint area
        Cursor.Position = new Point(random.Next(10, 930), random.Next(150, 880));
        Thread.Sleep(100);
    }
    // Release the left click
    SendMessage(childWindow, MOUSEEVENTF_LEFTUP, new IntPtr(MK_LBUTTON), CreateLParam(random.Next(10, 930), random.Next(150, 880)));
}

I got this code for the click simulation from here.

The click gets simulated but it doesn't paint anything. It seems that the click doesn't work inside MSPaint. The cursor changes to the "cross" of MSPaint but as I mentioned...the click doesn't seem to work.

FindWindow sets the value of hwndMain to value 0. Changing the parameter mspaint to MSPaintApp doesn't change anything. The value of hwndMain stays 0.

If it helps, here is my OpenPaint() method:

private static void OpenPaint()
{
    Process.process = new Process();
    process.StartInfo.FileName = "mspaint.exe";
    process.StartInfo.WindowStyle = "ProcessWindowStyle.Maximized;
    process.Start();
}

What am I doing wrong?

Answer 1

IntPtr hwndMain = FindWindow("mspaint", null);

That isn't good enough. Common mistake in pinvoke code, C# programmers tend to rely entirely too much on an exception to jump off the screen and slap them in the face to tell them that something went wrong. The .NET Framework does do that extra-ordinarily well. But that does not work the same way when you use an api that's based on the C language, like the winapi. C is a dinosaur language and did not support exceptions at all. It still doesn't. You'll only get an exception when the pinvoke plumbing failed, usually because of the bad [DllImport] declaration or a missing DLL. It does not speak up when the function executed successfully but returns a failure return code.

That does make it entirely your own job to detect and report failure. Just turn to the MSDN documentation, it always tells you how a winapi function indicates a mishap. Not completely consistent, so you do have to look, in this case FindWindow returns null when the window could not be found. So always code it like this:

IntPtr hwndMain = FindWindow("mspaint", null);
if (hwndMain == IntPtr.Zero) throw new System.ComponentModel.Win32Exception();

Do this for all the other pinvokes as well. Now you can get ahead, you'll reliably get an exception instead of plowing on with bad data. Which, as is so often the case with bad data, isn't quite bad enough. NULL is actually a valid window handle, the OS will assume you meant the desktop window. Ouch. You are automating the completely wrong process.

---

Understanding why FindWindow() fails does require a bit of insight, it is not very intuitive, but good error reporting is crucial to get there. The Process.Start() method only ensures that the program got started, it does not in any way wait until the process has completed its initialization. And in this case, it does not wait until it has created its main window. So the FindWindow() call executes about, oh, a couple of dozen milliseconds too early. Extra puzzling since it works just fine when you debug and single-step through the code.

Perhaps you recognize this kind of mishap, it is a threading race bug. The most dastardly kind of programming bug. Infamous for not causing failure consistently and very hard to debug since races are timing dependent.

Hopefully you realize that the proposed solution in the accepted answer is not good enough either. Arbitrarily adding Thread.Sleep(500) merely improves the odds that you now wait long enough before calling FindWindow(). But how do you know that 500 is good enough? It is always good enough?

No. Thread.Sleep() is never the correct solution for a threading race bug. If the user's machine is slow or is too heavily loaded with a shortage of available unmapped RAM then a couple of milliseconds turns into seconds. You have to deal with the worst case, and it is worst indeed, only ~10 seconds is in general the minimum you need to consider when the machine start thrashing. That's getting very unpractical.

Interlocking this reliably is such a common need that the OS has a heuristic for it. Needs to be a heuristic instead of a WaitOne() call on a synchronization object since the process itself doesn't cooperate at all. You can generally assume that a GUI program has progressed sufficiently when it starts asking for notifications. "Pumping the message loop" in Windows vernacular. That heuristic also made it into the Process class. Fix:

private static void OpenPaint()
{
    Process.process = new Process();
    process.StartInfo.FileName = "mspaint.exe";
    process.StartInfo.WindowStyle = "ProcessWindowStyle.Maximized;
    process.Start();
    process.WaitForInputIdle();          // <=== NOTE: added
}

I would be remiss if I didn't point out that you should use the built-in api for this. Called UI Automation, wrapped ably in the System.Windows.Automation namespace. Takes care of all those nasty little details, like threading races and turning error codes into good exceptions. Most relevant tutorial is probably here.