How to capture data in a window

Question

I have a desktop application installed on my machine. When I start a program some kind of window gets open. let's say, something like this (just example):

So, I want to write an application in C# that will find this window and capture some data from it.

What tools should I look at? I want to go with a path of least resistance.

I need to capture images, text from textboxes, and also find controls by text and click on them.

Answer 1

I suggest you use the cool but little-known UI Automation API for this work.

For this, the first thing to test is launch the associated UISpy tool. It will display a tree of all accessible windows on screen. It also is able to run some actions like pressing a menu, selecting an item, etc. This is using what's called UI Automation Control Patterns, which provide a way to categorize and expose a control's functionality independent of the control type or the appearance of the control.

So, if you can automate this application with UI Spy, you also can do the exact same thing using .NET code (UISpy is itself simply using the underlying API).

Here is an interesting tutorial article about UI automation programming: The Microsoft UI Automation Library

Answer 2

You should start enumerating handles of all windows for that process :

https://stackoverflow.com/a/2584672/351383

Then for each handle get information about text and position, with position infomation you can take screenshots of desktop on that position to get images AFAIK there is no other way to get images from a window of running application.

When you got screen positions of the controls then use from link below to simulate left mouse click, search windows for some text and then click on some point inside control, here is the method that will click a point :

https://stackoverflow.com/a/10355905/351383

I put toghether quick class to gather that data for process :

 public static class ProcessSpy
  {
    public static List<ProcessSpyData> GetDataForProcess(string processName)
    {
      var result = new List<ProcessSpyData>();
      Process myProc = Process.GetProcessesByName(processName).FirstOrDefault();
      if (myProc != null)
      {
        var myHandles = EnumerateProcessWindowHandles(myProc);
        foreach (IntPtr wndHandle in myHandles)
        {
          result.Add(new ProcessSpyData(wndHandle));
        }
      }
      return result;
    }

    delegate bool EnumThreadDelegate(IntPtr hWnd, IntPtr lParam);

    [DllImport("user32.dll")]
    static extern bool EnumThreadWindows(int dwThreadId, EnumThreadDelegate lpfn, IntPtr lParam);

    static IEnumerable<IntPtr> EnumerateProcessWindowHandles(Process prc)
    {
      var handles = new List<IntPtr>();

      foreach (ProcessThread thread in prc.Threads)
        EnumThreadWindows(thread.Id, (hWnd, lParam) => { handles.Add(hWnd); return true; }, IntPtr.Zero);

      return handles;
    }
  }

  public class ProcessSpyData
  {
    private const uint WM_GETTEXT = 0x000D;

    [DllImport("user32.dll", CharSet = CharSet.Auto)]
    static extern IntPtr SendMessage(IntPtr hWnd, uint Msg, int wParam, StringBuilder lParam);

    [DllImport("user32.dll")]
    private static extern bool GetClientRect(IntPtr hWnd, out RECT lpRect);

    [StructLayout(LayoutKind.Sequential)]
    private struct RECT
    {
      int left, top, right, bottom;

      public Rectangle ToRectangle()
      {
        return new Rectangle(left, top, right - left, bottom - top);
      }
    }

    [DllImport("user32.dll")]
    static extern bool ClientToScreen(IntPtr hWnd, ref Point lpPoint);

    public IntPtr WindowHandle { get; private set; }
    public string WindowText { get; private set; }
    public Rectangle ClientRect { get; private set; }
    public Rectangle ScreenPos { get; private set; }

    public ProcessSpyData(IntPtr windowHandle)
    {
      this.WindowHandle = windowHandle;
      GetWindowText();
      GetWindowSize();
    }

    private void GetWindowText()
    {
      StringBuilder message = new StringBuilder(1024);
      SendMessage(this.WindowHandle, WM_GETTEXT, message.Capacity, message);
      this.WindowText = message.ToString();
    }

    private void GetWindowSize()
    {
      var nativeRect = new RECT();
      GetClientRect(this.WindowHandle, out nativeRect);
      this.ClientRect = nativeRect.ToRectangle();

      Point loc = this.ClientRect.Location;
      ClientToScreen(this.WindowHandle, ref loc);
      this.ScreenPos = new Rectangle(loc, this.ClientRect.Size);
    }
  }

That should get you started, but you have to be aware if app is using non standard controls then there is no way to get text out of it with this method, and for images maybe you will get better results looking at executable resources.

UPDATE

Geting controls text for various control types (MFC, winforms, Delphi VCL etc.) would be very hard task, but for winforms see excelent Managed Windows API, they even have some sort of spy application in tools, look at that.