Invoke on click on a button using UI Automation with no InvokePattern or clickablePoint

Question

i'm try to send click message to (or invoke) a button in another application .

i used UISpy.exe and could find the element which i need.

but it has no id,no clickablePoint and no Invoke pattern.

i tried following code:

var processStartInfo = new ProcessStartInfo(@"tdesktop\Program.exe");
        var proc = Process.Start(processStartInfo);
        Thread.Sleep(3000);
        AutomationElement mainWin = AutomationElement.RootElement.FindChildByProcessId(proc.Id);
        List<AutomationElement> elmList= GetChildren(mainWin);
        //MessageBox.Show(elmList.Count.ToString());
        if (elmList.Count == 7)
        {
           List<AutomationElement> menubar= GetChildren(elmList[6]);

           AutomationElement elementNode = menubar[1];
           double x = elementNode.GetClickablePoint().X;
           double y = elementNode.GetClickablePoint().Y;

           win32 w = new win32();
           w.move_left_click((UInt32)x, (UInt32)y);


        }

it throws an exception in elementNode.GetClickablePoint().X that the Autumation Element has no clickable point.

i tried also TryGetInvokePattern() but still throws execption it has no InvokePattern.

i use VS2012 and .net 4.5

is there any way to make this?

Answer 1

As was already suggested, I'd strongly recommend pointing the Inspect SDK tool to the UI you're interested in. The tool (inspect.exe) be found in places like "C:\Program Files (x86)\Windows Kits\8.1\bin\x64".

By using the tool, you can see how the UI element of interest is being exposed programmatically. For example, is it exposed as a single UIA element, or it is part of a larger UIA element which represents a set of UI elements shown visually, and what UIA patterns does it support? As a test, I just pointed Inspect to an arrow shape in Paint. The results are shown below.

So I can tell that the arrow is exposed programmatically as a single UIA element, and that it supports the Invoke pattern. This means I can programmatically invoke the button through UIA. (And I can call the pattern methods on the element from inside Inspect’s Action menu, which is pretty handy.) If the UIA element didn't support any patterns that would allow me to programmatically control it, I can find its BoundingRectangle property through UIA, and simulate a mouse click in the middle of that to invoke it. (And I'm assuming the button's not obscured when I simulate the mouse click.)

But if I look at another group of elements shown visually on the screen, using Inspect I can learn that the whole set is exposed through UIA as a single UIA element. So in the image of Inspect shown below, I can learn that I'm not able to programmatically invoke a specific color in that group.

So in that case, I'd probably have to assume I know the size and layout of the UI elements shown visually in that group, and simulate a mouse click at some point which I think is appropriate based on the color I want to invoke.

By using Inspect, I can get a good understanding of what my options are. Ideally a single element shown visually on the screen will be exposed through UIA as a single element that I can control through whatever patterns are relevant, (for example, Invoke, Toggle, SelectionItem etc). But if useful patterns aren't supported, then I could consider simulating mouse clicks instead, based on whatever ClickablePoint or BoundingRectangle data's exposed.

Thanks,

Guy

Answer 2

A menu bar doesn't expose the InvokePattern (see UI Automation Support for the MenuBar Control Type). However, a menu item can be Invoked (see UI Automation Support for the MenuItem Control Type).

The following code illustrates how to generate a list of menu items:

AutomationElementCollection items = menubar.FindAll(
    TreeScope.Children,
    new PropertyCondition(AutomationElement.ControlTypeProperty, ControlType.MenuItem));