Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Windows UI Automation doesn't recognize button controls

I'm having problems trying to identify via Windows UI Automation the button controls that are inside the Notification Area window (classname: ToolbarWindow32):

enter image description here

I verified via the Windows UI Automation tools deployed in the Windows SDK that those "icons" are controls of type ControlType.Button, however when I try to run the code below I get a null-reference exception because the search condition I use doesn't get any control.

I'm doing something wrong, or maybe I found some kind of limitation in Windows UI Automation ?

This is the code, I mixed it with WinAPI calls just to facilitate the task for the helper users who maybe preffers to use that methodology.

Dim tskBarClassName As String = "Shell_TrayWnd"
Dim tskBarHwnd As IntPtr = NativeMethods.FindWindow(tskBarClassName, Nothing)

Dim systrayBarClassName As String = "TrayNotifyWnd"
Dim systrayBarHwnd As IntPtr = NativeMethods.FindWindowEx(tskBarHwnd, IntPtr.Zero, systrayBarClassName, Nothing)

Dim ntfyBarClassName As String = "ToolbarWindow32"
Dim ntfyBarHwnd As IntPtr = NativeMethods.FindWindowEx(systrayBarHwnd, IntPtr.Zero, ntfyBarClassName, Nothing)

Dim window As AutomationElement = AutomationElement.FromHandle(ntfyBarHwnd)
Dim condition As New PropertyCondition(AutomationElement.ControlTypeProperty, ControlType.Button)
Dim button As AutomationElement = window.FindFirst(TreeScope.Descendants, condition)

MsgBox(button.Current.Name) ' Here throws the null-reference exception.

Any solution for this?

like image 514
ElektroStudios Avatar asked Sep 22 '16 18:09

ElektroStudios


1 Answers

I verified via the Windows UI Automation tools deployed in the Windows SDK that those "icons" are controls of type ControlType.Button

You are correct somewhat. Technically they are not in ToolbarWindow32, rather Shell_TrayWnd. I inspected the area and found out, those buttons are actually in a ToolBar, so you need to look for the ControlType.ToolBar. Next you need to FindAll which will return all AutomationElements that would satisfy the PropertyCondition...

Note: the first loop is to get the User Promoted Notification Area. The next loop for fun is getting the Running Application buttons... (CODE IS WORKING ON WIN7, WIN8 and WIN10)

Here in my example below, I go after the Shell_TrayWnd which will get us what we would need. Then I go through and find what ToolBar we are after, then loop through and FindAll ControlTypes of Button...

Dim arrText As New List(Of String)
        Dim tskBarClassName As String = "Shell_TrayWnd"
        Dim tskBarHwnd As IntPtr = FindWindow(tskBarClassName, Nothing)
        Dim window As AutomationElement = AutomationElement.FromHandle(tskBarHwnd)
        Dim condition As New PropertyCondition(AutomationElement.ControlTypeProperty, ControlType.ToolBar)
        Dim elementCollection As AutomationElementCollection = window.FindAll(TreeScope.Descendants, condition)

        'for fun get all we can...
        For Each aE As AutomationElement In elementCollection
            If aE.Current.Name.Equals("User Promoted Notification Area") Then
                For Each ui As AutomationElement In aE.FindAll(TreeScope.Descendants, New PropertyCondition(AutomationElement.ControlTypeProperty, ControlType.Button))
                    arrText.Add("Notification Area - " & Replace(ui.Current.HelpText, Chr(10), " "c)) 'removed line break as when shown it would show some on a new line in messagebox
                Next
            ElseIf aE.Current.Name.Equals("Running applications") Then
                For Each ui As AutomationElement In aE.FindAll(TreeScope.Descendants, New PropertyCondition(AutomationElement.ControlTypeProperty, ControlType.Button))
                    arrText.Add("Toolbar Area - " & Replace(ui.Current.Name, Chr(10), " "c)) 'removed line break as when shown it would show some on a new line in messagebox
                Next
            End If

        Next

If arrText.Count > 0 Then
            MessageBox.Show(String.Join(Environment.NewLine, arrText.ToArray))
        End If

If you have any questions, please let me know. Image below (some things I commented out for secure reasons)

enter image description here

like image 105
Trevor Avatar answered Sep 18 '22 19:09

Trevor