WCF : maxConcurrentCalls has exhausted

Question

I'm in the process of troubleshooting a WCF Service that hangs at some point. The service behavior is the following:

[ServiceBehavior( InstanceContextMode = InstanceContextMode.PerCall, ConcurrencyMode = ConcurrencyMode.Multiple )]

Throttling parameters are :

<serviceThrottling maxConcurrentCalls="50" maxConcurrentSessions="50" maxConcurrentInstances="50" />

Following is the service state taken from a hang dump :

0:000> !mdt 0000000000c9f270 -r
0000000000c9f270 (System.ServiceModel.Dispatcher.ServiceThrottle)
   calls:0000000000c9f3d8 (System.ServiceModel.Dispatcher.FlowThrottle)
      capacity:0x32 (System.Int32)
      count:0x32 (System.Int32)
      mutex:0000000000c9f418 (System.Object)
         <NO FIELDS>
      release:0000000000c9f398 (System.Threading.WaitCallback)
         _target:0000000000c9f270 (System.ServiceModel.Dispatcher.ServiceThrottle)
            <RECURSIVE>
         _methodBase:NULL (System.Reflection.MethodBase)
         _methodPtr:0000064273dddf30 (System.IntPtr)
         _methodPtrAux:0000000000000000 (System.IntPtr)
         _invocationList:NULL (System.Object)
         _invocationCount:0000000000000000 (System.IntPtr)
         <NO FIELDS>
      waiters:0000000000c9f430 (System.Collections.Generic.Queue`1[[System.Object, mscorlib]])
         _array:0000000028d73e70 (System.Object[], Elements: 16)
         _head:0x1 (System.Int32)
         _tail:0xA (System.Int32)
         _size:0x9 (System.Int32)
         _version:0x22 (System.Int32)
         _syncRoot:NULL (System.Object)
      propertyName:0000000000c9f2b8 (System.String: "MaxConcurrentCalls")
      configName:0000000000c9f358 (System.String: "maxConcurrentCalls")
   sessions:0000000000c9f508 (System.ServiceModel.Dispatcher.FlowThrottle)
      capacity:0x32 (System.Int32)
      count:0x9 (System.Int32)

....

   instanceContexts:000000000105ffc8 (System.ServiceModel.Dispatcher.FlowThrottle)
      capacity:0x32 (System.Int32)
      count:0x32 (System.Int32)

As you can see, maxConcurrentCalls has exhausted, while sessions count is only 9. I'm wondering if this could be caused by a malfunction in client code about proxy usage, like poor exception handling ?

Given a memory dump of the service, is there a way to find clients IP adresses ?

TIA.

Answer 1

Found the source of the problem. MaxConcurrentCalls has exhausted because of a deadlock in the service code. We are using the c# lock keyword and it seems that sometimes the lock is not released when something weird happens in the code protected by the lock...

Anyway, thanks to everyone that has contributed to this thread.