Why is it taking so long to GC System.Threading.OverlappedData?

Question

I'm running my application through a memory profiler to check for leaks. Things seem to be sort of OK, but I'm getting a lot of these OverlappedData that seems to be hanging around in the finalizer queue doing next to nothing. They are the result of overlapped IO that has been cancelled by shutting down the underlying NetworkStream on either end of the connection.

The network stream itself is disposed. There are no live instances of NetworkStream anywhere.

Typically they're rooted in something that is called an OverlappedDataCacheLine.I'm calling EndRead in a callback the first thing I do, so no call to BeginRead should be without it's corresponding EndRead.

This is a pretty typical look of who's keeping it from the tool

In the end it does get GC'd but it takes forever - in the order of half an hour to kill everything when I've started about a thousand streams, put them in a async call to BeginRead and shutting them down after about a minute.

This program reproduces the problem somewhat against a webserver on port 80. Any webserver will do really.

using System;
using System.Collections.Generic;
using System.Net.Sockets;
using System.Threading;

class Program
{
    static void Main(string[] args)
    {
        var clients = new List<TcpClient>();
        for (int i = 0; i < 1000; ++i) {
            var client = new TcpClient();
            clients.Add(client);

            client.BeginConnect("localhost", 80, connectResult =>
                {
                    client.EndConnect(connectResult);

                    var stream = client.GetStream();
                    stream.BeginRead(new byte[1000], 0, 1000, result =>
                        {
                            try
                            {
                                stream.EndRead(result);
                                Console.WriteLine("Finished (should not happen)");
                            }
                            catch
                            {
                                // Expect to find an IO exception here
                                Console.WriteLine("Faulted");                               
                            }
                        }, stream);     
                }, client);             
        }

        Thread.Sleep(10000); // Make sure everything has time to connect

        foreach (var tcpClient in clients)
        {
            tcpClient.GetStream().Close();
            tcpClient.Close();
        }
        clients.Clear(); // Make sure the entire list can be GC'd

        Thread.Sleep(Timeout.Infinite); // Wait forever. See in profiler to see the overlapped IO take forever to go away
    }
}

Granted, this program doesn't take forever to clear up the thousand OverlappedData since it's way smaller than the real application but it does take a while to do its thing. I get warnings for a stuck finalizer when running my real stuff instead of this testing application. It doesn't do much in my application, just tries to close down everything that might not have been closed and makes sure that there's no references being kept to anything anywhere.

It doesn't seem to matter at all if I call Dispose() or Close() on the client and it's stream. The result is the same.

Any clue as to why this happens and how to avoid this? It the CLR being smart on me and keeping these pinned blocks of memory intact in preparation for new calls perhaps? And why is the finalizer being so incredibly slow to complete?

Update After doing some incredibly stupid load tests by putting a glass of water on the F5 key and getting some coffee it seems that something triggers a more complete GC under stress that collects these things. So there doesn't actually seem to be a real issue but still it would be nice to know what actually goes on here and why collecting this object is magnitudes slower than other objects and if this could potentially be an issue at a later stage with fragmented memory and such.

Answer 1

Okay, it seems clear now what's going on. Garbage collections only ever happen when you allocate memory. It requires at least 2 megabytes of allocations, the typical initial size of the generation 0 GC heap to trigger a GC. In other words, a program that does nothing will never run a GC and you'll see any objects that haven't been collected yet in the heap with a memory profiler for a long time.

Which is a good explanation of what you describe. After you terminate all connections, your program doesn't have to do anything anymore. So won't allocate much if any memory so won't trigger a collection. If your profiler doesn't show collections then you can see them with Perfmon.exe. This is otherwise not a problem at all, just a side effect of how a garbage collector works.

Only ever worry about leaks when you have clear evidence that a program has a run-away resource consumption problem.