AnonymousPipeServerStream.Read() occasionally hangs on client exit

Question

I have a master and a slave program who interact through a pair of anonymous pipes.

The interaction looks like this:

• Master creates two AnonymousPipeServerStream's
• Master starts client process, giving him their .GetClientHandleAsString()
• Master .DisposeLocalCopyOfClientHandle's both pipes
• Master writes stuff to one pipe and reads stuff from the other one
• Occasionally master forcefully terminates the slave from another thread (Process.Kill()), AND ALSO closes both pipe objects.

In very rare circumstances, upon slave termination, reading from the pipe on master's side blocks indefinitely - or sometimes definitely (e.g. for several minutes). I am not able to reproduce it on my local machine, but it happens from time to time on a large cluster.

I captured a dump of this situation, and here's what I saw:

Stacktrace of blocked master (I am 100% sure that at this moment the client process is already terminated):

000000000c83e488 000000007700fdba [NDirectMethodFrameStandalone: 000000000c83e488] Microsoft.Win32.UnsafeNativeMethods.ReadFile(Microsoft.Win32.SafeHandles.SafePipeHandle, Byte*, Int32, Int32 ByRef, IntPtr)
000000000c83e430 000007feeab32820 DomainBoundILStubClass.IL_STUB_PInvoke(Microsoft.Win32.SafeHandles.SafePipeHandle, Byte*, Int32, Int32 ByRef, IntPtr)*** WARNING: Unable to verify checksum for System.Core.ni.dll

000000000c83e540 000007feeac14574 System.IO.Pipes.PipeStream.ReadFileNative(Microsoft.Win32.SafeHandles.SafePipeHandle, Byte[], Int32, Int32, System.Threading.NativeOverlapped*, Int32 ByRef)
000000000c83e5a0 000007feeac14a23 System.IO.Pipes.PipeStream.ReadCore(Byte[], Int32, Int32)
000000000c83e610 000007fef0169d8f System.IO.BinaryReader.FillBuffer(Int32)
000000000c83e650 000007fef0169c8a System.IO.BinaryReader.ReadInt32()

I also looked at the AnonymousPipeServerStream object that's blocking, and at its state and handle.

It has:

• m_state = 4 (i.e. closed)
• m_clientHandle is a closed handle (according to SafeHandle._state and to the output of !handle)
• m_handle is an open handle, with _state = 6 (i.e. NOT closed, even though the Pipe object is closed and, according to decompilation, it should have called m_handle.Dispose()):

Here's the output of !handle for the master-side handle.

0:000> !handle 1850 ff
Handle 0000000000001850
  Type          File
  Attributes    0
  GrantedAccess 0x120189:
         ReadControl,Synch
         Read/List,ReadEA,ReadAttr,WriteAttr
  HandleCount   2
  PointerCount  5
  No object specific information available

I am a bit worried that the HandleCount is 2 and PointerCount is 5.

Any ideas what could have gone wrong? Why was the master's end of the reading handle not closed when I closed the pipe? Why was the pipe not broken, despite the client termination and despite the fact I called DisposeLocalCopyOfClientHandle?

What can I do to at least work around this? Maybe just do Thread.Interrupt on the reading thread?..

Answer 1

Possible sequence of events in master process, preceded by thread numbers:

• 1 - About to call Microsoft.Win32.UnsafeNativeMethods.ReadFile (inside BinaryReader.ReadInt32)
• 2 - Kill slave process and close pipe handles
• 3 - The win32 pipe handle (which is now closed) is reused for some other pipe (or file or whatever)
• 1 - Microsoft.Win32.UnsafeNativeMethods.ReadFile starts reading using the old handle that now refers to some other object, and blocks.

Fix:

Kill the slave process in thread 2 but don't close the handles. This will cause any read in thread 1 to terminate, then close the pipe handles in thread 1.

Moral:

Don't close handles while you're still using them.