UIAutomation and Remote Desktop Connections

Question

I have an application that does some automation on an arbitrary Windows application. I've been using Notepad to test it.

During this automation, I have to invoke events from the code. I currently support two types of events since they seem to be the most common, InvokePattern and ExpandCollapsePattern.

I use a computer that I can remote desktop into to do the testing. But it's weird. When I'm connected to the computer through Remote Desktop, the application works fine. When I disconnect from the computer, my code stops working for the ExpandCollapsePattern (InvokePattern works fine). I get an InvalidOperationException.

The documentation says that this should only be thrown if the node is marked as a LeafNode. What makes Remote Desktop different that I'm getting the InvalidOperationException?

Here is the code I currently have to perform the event.

ExpandCollapseState state =
    patternMenu.Current.ExpandCollapseState;
if (state == ExpandCollapseState.Expanded)
    patternMenu.Collapse();
else if (state == ExpandCollapseState.PartiallyExpanded ||
         state == ExpandCollapseState.Collapsed)
    patternMenu.Expand();

patternMenu is an ExpandCollapsePattern gotten from the AutomationElement using GetCurrentPattern.

The current value of ExpandCollapseState is "Collapsed" when I print out the value.

EDIT: Any chance I can know why I got a downvote and how this is a bad question? I'm genuinely confused of what's happening, as it only seems to fail when remote desktop is closed. If this is a really dumb question that I should know the answer to, I'd love an explanation and then a downvote.

The process that interacts with the desktop isn't a Windows service. It's an application I start after I remote desktop into the computer. Is it because I'm locking the desktop?

I'm using "Terminal Server Client" on Ubuntu to log into the Windows machine. Does pressing the close button on this application cause the desktop to lock?

Answer 1

Generally speaking, you can't typically run UI Automation or similar on a non-interactive desktop. Non-interactive desktops have various limitations regarding input: there's no focused element, input can't be sent, so anything that depends on these will fail.

Some functionality, such as sending windows messages, will work fine. What might be happening here is that the InvokePattern functionality is implemented under the covers as messages, so still works; but the Expand/Collapse might be implemented in terms of input, which would fail.

Not clear why, in your case, it fails when you close the client (that part is expected), but appears to work when the remote desktop locks itself - would it expect to behave the same in both cases.

(Note that if the local Ubuntu desktop locks, all should be fine and still work, so long as the client still running. The remote client is still 'interactive' in that case, since it's got a live client attached to it, regardless of whether the client itself is running on an interactive or non-interactive desktop - the concepts may not even apply to a client running on another OS!)