How to prevent re-entrancy of WPF event handler during ActiveX method call?

Question

We're calling methods on an ActiveX component from within a WPF and STA application. This calling is late-bound performed via:

res = ocx.GetType().InvokeMember(methodName, flags, null, ocx, args);

...where ocx is the ActiveX object retrieved with the System.Windows.Forms.AxHost.GetOcx() method.

This call is performed from within a WPF event handler, say 'mouse clicked'.

Now the problem. If we double-click the 'mouse clicked' event will trigger, running InvokeMember(). However, during this call, we see that the 'mouse clicked' event is re-entered. So in the same thread, we see the event handler twice on the call stack. This is very unexpected, and we're trying to prevent that. How can we prevent this from happening?

The only reason we can think of why it happens is:

• The COM object is created in another STA, so we're performing a cross-STA call which needs to be marshalled
• cross-thread STA calls uses a Windows Message to send an RPC request to the COM component
• cross-thread STA calls use the Windows Message pump to receive RPC answer
• During the waiting another type of event comes in (like 'mouse clicked'), and this gets handled before the RPC answer gets handled.
• This RPC answer gets handled

Things we tried to fix the problem:

• use lock() in all event handlers. This does not work since lock() will lock a thread, and in this case it is the same thread which re-enters the event handler.
• use custom locking like 'bool locked = false; if (!locked) { locked = true; InvokeMethod(); ...; locked = false; }'. This works partially: it throws away the events instead of queuing them for later, and need an extensive change to all our event handlers, which is not nice to do.
• use Dispatcher.DisableProcessing to stop (other) messages from being processed. This does not help: it throws an exception because of messages being processed anyways.
• create a second dispatcher in a new thread, and run ocx.InvokeMehod() via Dispatcher.Invoke() to have it handled by another thread. This gives 'An event was unable to invoke any of the subscribers (Exception from HRESULT: 0x80040201)' (Yes, we're also subscribed to COM events of the ActiveX object).
• use Dispatcher.PushFrame() to stop event handling from happening. This also fails.

A wild idea which might work, but don't know how to implement this would be creating a new message pump as the WPF message pump which can be configured to temporarily handle only RPC calls. This is along the lines of http://jmorrill.hjtcentral.com/Home/tabid/428/EntryId/430/WPF-MediaKit-Updates.aspx , but still somewhat different from this situation.

So the question boils down to how can we make ActiveX call synchronously like we expected it already to be instead of async?

Update

To make it more clear that the mechanism involved is not only about mouse-events, but the more generic problem of 'a new event is handled while the old is being executed', I'll give another example with a stack trace:

Context: we've got a WPF Grid on which we get a mouseclick (Grid_MouseDown), we've got an ActiveX object on which we perform the method 'CloseShelf'. Opening the shelf will take time, so we are subscribed to the event 'EventShelfClosed', which in the event handler of EventShelfClosed will call 'ListShelf' to know which shelfs are left.

This is how the managed stack trace looks like (Hans asked for an unmanaged stacktrace, but I don't know how to get one):

MyAxWrapper.dll!MyAxWrapper.MyAxWrapper.InvokeMember(string methodName, System.Reflection.BindingFlags flags, object[] args, int refArgIdx) Line 53 C#
MyAxWrapper.dll!MyAxWrapper.LoggingMyAxWrapper.InvokeMember(string methodName, System.Reflection.BindingFlags flags, object[] args, int refArgIdx) Line 151 + 0x14 bytes    C#
MyAxWrapper.dll!MyAxWrapper.MyAxWrapper.InvokeMethod(string methodName, object[] args) Line 92 + 0x18 bytes C#
MyAxWrapper.dll!MyAxWrapper.MyAxAppWrapper.ListShelfs(string CanvasPageId) Line 300 + 0x42 bytes    C#
PACS.dll!PACS.MyAxDatabase.GetShelfIdsOn(string canvasPageId) Line 223 + 0xf bytes  C#
MyAxCanvas.dll!MyAxCanvas.MyAxCanvasPlugin.UpdateTimeLineSelection(string canvasPageId) Line 123 + 0x10 bytes   C#
MyAxCanvas.dll!MyAxCanvas.MyAxCanvasPlugin.EventShelfClosed(string canvasPageId, string shelfId) Line 180 + 0xb bytes   C#
MyAxWrapper.dll!MyAxWrapper.MyAxAppWrapper.FireEvent(string eventName, object[] args) Line 21 + 0x73 bytes  C#
MyAxWrapper.dll!MyAxWrapper.MyAxEventForwarder.EventShelfClosed(string CanvasPageID, string ShelfID) Line 177 + 0x58 bytes  C#
[Native to Managed Transition]  
[Native to Managed Transition]  
MyAxWrapper.dll!MyAxWrapper.MyAxWrapper.InvokeMember(string methodName, System.Reflection.BindingFlags flags, object[] args, int refArgIdx) Line 75 + 0x2b bytes    C#
MyAxWrapper.dll!MyAxWrapper.LoggingMyAxWrapper.InvokeMember(string methodName, System.Reflection.BindingFlags flags, object[] args, int refArgIdx) Line 151 + 0x14 bytes    C#
MyAxWrapper.dll!MyAxWrapper.MyAxWrapper.InvokeMethod(string methodName, object[] args) Line 92 + 0x18 bytes C#
MyAxWrapper.dll!MyAxWrapper.MyAxAppWrapper.CloseShelf(string a) Line 218 + 0x42 bytes   C#
MyAxCanvas.dll!MyAxCanvas.MyAxCanvasPlugin.EventCanvasPageCreated.AnonymousMethod__0(DataModel.Item exam) Line 110 + 0x1d bytes C#
ItemPresenter.dll!ItemPresenter.ItemPresenter.OnItemClicked(DataModel.Item study) Line 36 + 0x14 bytes  C#
ItemPresenter.dll!ItemPresenter.ItemPresenter.ItemPresenterPerYearControls_Click(object sender, System.Windows.RoutedEventArgs e) Line 215 + 0x1e bytes C#
PresentationCore.dll!System.Windows.RoutedEventHandlerInfo.InvokeHandler(object target, System.Windows.RoutedEventArgs routedEventArgs) + 0x78 bytes    
PresentationCore.dll!System.Windows.EventRoute.InvokeHandlersImpl(object source, System.Windows.RoutedEventArgs args, bool reRaised) + 0x1ae bytes  
PresentationCore.dll!System.Windows.UIElement.RaiseEventImpl(System.Windows.DependencyObject sender, System.Windows.RoutedEventArgs args) + 0x79 bytes  
PresentationCore.dll!System.Windows.UIElement.RaiseEvent(System.Windows.RoutedEventArgs e) + 0x17 bytes 
ItemPresenter.dll!ItemPresenter.ItemPresenterControl.Grid_MouseDown(object sender, System.Windows.Input.MouseButtonEventArgs e) Line 47 + 0x29 bytes    C#
PresentationCore.dll!System.Windows.Input.MouseButtonEventArgs.InvokeEventHandler(System.Delegate genericHandler, object genericTarget) + 0x31 bytes    
PresentationCore.dll!System.Windows.RoutedEventArgs.InvokeHandler(System.Delegate handler, object target) + 0x29 bytes  
PresentationCore.dll!System.Windows.RoutedEventHandlerInfo.InvokeHandler(object target, System.Windows.RoutedEventArgs routedEventArgs) + 0x3e bytes    
PresentationCore.dll!System.Windows.EventRoute.InvokeHandlersImpl(object source, System.Windows.RoutedEventArgs args, bool reRaised) + 0x1ae bytes  
PresentationCore.dll!System.Windows.UIElement.RaiseEventImpl(System.Windows.DependencyObject sender, System.Windows.RoutedEventArgs args) + 0x79 bytes  
PresentationCore.dll!System.Windows.UIElement.RaiseTrustedEvent(System.Windows.RoutedEventArgs args) + 0x41 bytes   
PresentationCore.dll!System.Windows.UIElement.RaiseEvent(System.Windows.RoutedEventArgs args, bool trusted) + 0x2c bytes    
PresentationCore.dll!System.Windows.Input.InputManager.ProcessStagingArea() + 0x1ff bytes   
PresentationCore.dll!System.Windows.Input.InputManager.ProcessInput(System.Windows.Input.InputEventArgs input) + 0x45 bytes 
PresentationCore.dll!System.Windows.Input.InputProviderSite.ReportInput(System.Windows.Input.InputReport inputReport) + 0x62 bytes  
PresentationCore.dll!System.Windows.Interop.HwndMouseInputProvider.ReportInput(System.IntPtr hwnd, System.Windows.Input.InputMode mode, int timestamp, System.Windows.Input.RawMouseActions actions, int x, int y, int wheel) + 0x263 bytes 
PresentationCore.dll!System.Windows.Interop.HwndMouseInputProvider.FilterMessage(System.IntPtr hwnd, MS.Internal.Interop.WindowMessage msg, System.IntPtr wParam, System.IntPtr lParam, ref bool handled) + 0x46d bytes 
PresentationCore.dll!System.Windows.Interop.HwndSource.InputFilterMessage(System.IntPtr hwnd, int msg, System.IntPtr wParam, System.IntPtr lParam, ref bool handled) + 0x75 bytes   
WindowsBase.dll!MS.Win32.HwndWrapper.WndProc(System.IntPtr hwnd, int msg, System.IntPtr wParam, System.IntPtr lParam, ref bool handled) + 0xbe bytes    
WindowsBase.dll!MS.Win32.HwndSubclass.DispatcherCallbackOperation(object o) + 0x7d bytes    
WindowsBase.dll!System.Windows.Threading.ExceptionWrapper.InternalRealCall(System.Delegate callback, object args, int numArgs) + 0x53 bytes 
WindowsBase.dll!MS.Internal.Threading.ExceptionFilterHelper.TryCatchWhen(object source, System.Delegate method, object args, int numArgs, System.Delegate catchHandler) + 0x42 bytes    
WindowsBase.dll!System.Windows.Threading.Dispatcher.InvokeImpl(System.Windows.Threading.DispatcherPriority priority, System.TimeSpan timeout, System.Delegate method, object args, int numArgs) + 0xb4 bytes    
WindowsBase.dll!MS.Win32.HwndSubclass.SubclassWndProc(System.IntPtr hwnd, int msg, System.IntPtr wParam, System.IntPtr lParam) + 0x104 bytes    
[Native to Managed Transition]  
[Managed to Native Transition]  
WindowsBase.dll!System.Windows.Threading.Dispatcher.PushFrameImpl(System.Windows.Threading.DispatcherFrame frame) + 0xc1 bytes  
WindowsBase.dll!System.Windows.Threading.Dispatcher.PushFrame(System.Windows.Threading.DispatcherFrame frame) + 0x49 bytes  
WindowsBase.dll!System.Windows.Threading.Dispatcher.Run() + 0x4c bytes  
PresentationFramework.dll!System.Windows.Application.RunDispatcher(object ignore) + 0x17 bytes  
PresentationFramework.dll!System.Windows.Application.RunInternal(System.Windows.Window window) + 0x6f bytes 
PresentationFramework.dll!System.Windows.Application.Run(System.Windows.Window window) + 0x26 bytes 
PresentationFramework.dll!System.Windows.Application.Run() + 0x1b bytes 
MyAxCanvasStandalone.exe!MyAxCanvasStandalone.App.Main(string[] args) Line 37 + 0xa bytes   C#
[Native to Managed Transition]  
[Managed to Native Transition]  
mscorlib.dll!System.AppDomain.ExecuteAssembly(string assemblyFile, System.Security.Policy.Evidence assemblySecurity, string[] args) + 0x6d bytes    
Microsoft.VisualStudio.HostingProcess.Utilities.dll!Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly() + 0x2a bytes  
mscorlib.dll!System.Threading.ThreadHelper.ThreadStart_Context(object state) + 0x63 bytes   
mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool ignoreSyncCtx) + 0xb0 bytes    
mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state) + 0x2c bytes    
mscorlib.dll!System.Threading.ThreadHelper.ThreadStart() + 0x44 bytes   
[Native to Managed Transition]  

What happens is the the method 'CloseShelf' will close the shelf, but in this case 'CloseShelf' is so fast that the event 'EventShelfClosed' is emitted and handled during the call to CloseShelf. Now CloseShelf will call ListShelfs, but ListShelfs will fail and return null because the ActiveX component is locked by the 'CloseShelf' call which is still active.

Why is this a problem? Becasue the programmer does not expect a method call to be async. This hit us after creating a large program, which now means auditting all calls for unexpected behaviour.

What would we like to see in this case? We would like to see that 'CloseShelf' returns without handling other events during the call. The method should be synchronous, and any pending events handled during the main (non-recursive) message loop.

Having a 'lock' kind of boolean won't help here, since we would be missing events here, which locks up the application.

Answer 1

You have the answer!

Is it just an old hack? No, it's not, it's standard operating procedure when any type of re-entrancy is involved. This has worked flawlessly for me in more cases than I can remember, from humble single panel VB3 crud popups to huge MVVM/DDD wannable enterprise management apps.

It is what you mentioned: 'use custom locking like 'static bool locked = false; if (!locked) { locked = true; InvokeMethod(); ...; locked = false; }'.

EDIT

Note the comments from the OP. OK so this won't solve the problem! The 2nd event isn't a spurious click; it is a different event, critical to the proper operation of the system.

Please see my next answer for a few more attempts. #3 is the ugliest but should work.

Answer 2

I have dealt with similar problems in the past although not from WPF.

In a win32 application the recommended approach was to use IMessageFilter::MessagePending - this could be configured to say what types of messages were allowed to be handled when an outgoing STA call was already in progress. Here you had to be careful to ensure that any callbacks from your callee object were accepted and handled.

http://msdn.microsoft.com/en-us/library/windows/desktop/ms694352(v=vs.85).aspx

In WPF this is not available. I think your approach to using another thread is the correct way to go.

In principle you want your main thread to block on a child thread. The child thread can then make the outgoing COM call. You probably want to make the child thread an STA to avoid introducing other unknown issues. It is important that messages are pumped on the child thread and that any pointers or types are correctly marshalled as the child thread will be in a different COM apartment. Reentrancy is avoided because callbacks are the only thing that would attempt to message the thread that is pumping.

In WPF I believe that a Dispatcher should provide all the functionality that you need.

I'm not sure why your attempt to do this using a Dispatcher failed - it might be related to this known issue: http://support.microsoft.com/kb/926997