WPF interop deadlock on DisplaySettingsChanging

Question

I have a WPF application that hosts a modeless Win32 form. Everything runs smoothly, until I attach or detach VNC to the machine. Then the application deadlocks: It doesn't redraw anything anymore, doesn't react to user interaction. I've looked at the stack trace using WinDbg:

0012f03c 792b6865 System.Threading.WaitHandle.WaitOne(Int32, Boolean)
0012f050 7b6f1a4f System.Windows.Forms.Control.WaitForWaitHandle(System.Threading.WaitHandle)
0012f064 7ba2d68b System.Windows.Forms.Control.MarshaledInvoke(System.Windows.Forms.Control, System.Delegate, System.Object[], Boolean)
0012f104 7b6f33ac System.Windows.Forms.Control.Invoke(System.Delegate, System.Object[])
0012f138 7b920bd7 System.Windows.Forms.WindowsFormsSynchronizationContext.Send(System.Threading.SendOrPostCallback, System.Object)
0012f150 7a92ed62 Microsoft.Win32.SystemEvents+SystemEventInvokeInfo.Invoke(Boolean, System.Object[])
0012f184 7a92dc8f Microsoft.Win32.SystemEvents.RaiseEvent(Boolean, System.Object, System.Object[])
0012f1d0 7a92daec Microsoft.Win32.SystemEvents.OnDisplaySettingsChanging()
0012f1e0 7a574c9f Microsoft.Win32.SystemEvents.WindowProc(IntPtr, Int32, IntPtr, IntPtr)
0012f1e4 003c20dc [InlinedCallFrame: 0012f1e4] 
0012f3a8 57843a57 System.Windows.Threading.Dispatcher.PushFrameImpl(System.Windows.Threading.DispatcherFrame)
0012f3f8 57843129 System.Windows.Threading.Dispatcher.PushFrame(System.Windows.Threading.DispatcherFrame)
0012f404 578430cc System.Windows.Threading.Dispatcher.Run()
0012f410 55bed46e System.Windows.Application.RunDispatcher(System.Object)
0012f41c 55bec76f System.Windows.Application.RunInternal(System.Windows.Window)
0012f440 55bd3aa6 System.Windows.Application.Run(System.Windows.Window)
0012f450 55bd3a69 System.Windows.Application.Run()

Apparently, the VNC attach/detach raises an OnDisplaySettingsChanging event, which in turn tries to call some event using System.Windows.Forms.Control.Invoke, which sends a message to the main thread and then waits for a response. But since this all happens in the main thread, the message loop never gets the message and the wait never returns.

I've found a workaround using EnableSystemEventsThreadAffinityCompatibility (which essentially bypasses the Control.Invoke call), but it feels like a dirty hack.

Did anybody ever see something like this happen?

Does someone have a clue why the SystemEvents class would use Control.Invoke when the message arrives on the main (STA) thread (it does, I checked)?

EDIT: Answers to questions in comments:

• Does the same thing happen when changing display settings (e.g. res) without VNC? -> No.
• Does the same thing happen with a couple of different versions on VNC (including hte latest)? -> I've only tried the latest version 1.0.9.5.
• Any other details about the WPF app, controls, or Win32 contorls? -> There's a WPF main window and a modeless WinForms Form.

Answer 1

This is a program initialization problem. Watch out for custom splash screens. If the first event subscription doesn't happen on the main thread then it will fire notifications on the wrong thread. That can cause all kinds of nasty problems, ranging from odd painting artifacts to outright deadlock. Beware that several standard Winforms controls will use SystemEvents to know when they should repaint themselves when the Windows theme or system colors were changed.

One workaround is to explicitly subscribe a dummy event handler in the Main() method, before anything else is done.