Why does the use of StreamSocket in a loop cause a memory leak?

Question

I'm developing a C#, UWP 10 solution that communicates with a network device using a fast, continual read/write loop. The StreamSocket offered by the API seemed to work great, until I realized that there was a memory leak: there is an accumulation of Task<uint32> in the heap, in the order of hundreds per minute.

Whether I use a plain old while (true) loop inside an async Task, or using a self-posting ActionBlock<T> with TPL Dataflow (as per this answer), the result is the same.

I'm able to isolate the problem further if I eliminate reading from the socket and focus on writing: Whether I use the DataWriter.StoreAsync approach or the more direct StreamSocket.OutputStream.WriteAsync(IBuffer buffer), the problem remains. Furthermore, adding the .AsTask() to these makes no difference.

Even when the garbage collector runs, these Task<uint32>'s are never removed from the heap. All of these tasks are complete (RanToCompletion), have no errors or any other property value that would indicate a "not quite ready to be reclaimed".

There seems to be a hint to my problem on this page (a byte array going from the managed to unmanaged world prevents release of memory), but the prescribed solution seems pretty stark: that the only way around this is to write all communications logic in C++/CX. I hope this is not true; surely other C# developers have successfully realized continual high-speed network communictions without memory leaks. And surely Microsoft wouldn't release an API that only works without memory leaks in C++/CX

EDIT

As requested, some sample code. My own code has too many layers, but a much simpler example can be observed with this Microsoft sample. I made a simple modification to send 1000 times in a loop to highlight the problem. This is the relevant code:

public sealed partial class Scenario3 : Page
{
    // some code omitted

    private async void SendHello_Click(object sender, RoutedEventArgs e)
    {
        // some code omitted

        StreamSocket socket = //get global object; socket is already connected

        DataWriter writer = new DataWriter(socket.OutputStream);

        for (int i = 0; i < 1000; i++)
        {
            string stringToSend = "Hello";
            writer.WriteUInt32(writer.MeasureString(stringToSend));
            writer.WriteString(stringToSend);
            await writer.StoreAsync();
        }
    }
}

Upon starting up the app and connecting the socket, there is only instance of Task<UInt32> on the heap. After clicking the "SendHello" button, there are 86 instances. After pressing it a 2nd time: 129 instances.

Edit #2 After running my app (with tight loop send/receive) for 3 hours, I can see that there definitely is a problem: 0.5 million Task instances, which never get GC'd, and the app's process memory rose from an initial 46 MB to 105 MB. Obviously this app can't run indefinitly. However... this only applies to running in debug mode. If I compile my app in Release mode, deploy it and run it, there are no memory issues. I can leave it running all night and it is clear that memory is being managed properly. Case closed.

Answer 1

there are 86 instances. After pressing it a 2nd time: 129 instances.

That's entirely normal. And a strong hint that the real problem here is that you don't know how to interpret the memory profiler report properly.

Task sounds like a very expensive object, it has a lot of bang for the buck and a thread is involved, the most expensive operating system object you could ever create. But it is not, a Task object is actually a puny object. It only takes 44 bytes in 32-bit mode, 80 bytes in 64-bit mode. The truly expensive resource is not owned by Task, the threadpool manager takes care of it.

That means you can create a lot of Task objects before you put enough pressure on the GC heap to trigger a collection. About 47 thousand of them to fill the gen #0 segment in 32-bit mode. Many more on a server, hundreds of thousands, its segments are much bigger.

In your code snippet, Task objects are the only objects you actually create. Your for(;;) loop does therefore not nearly loop often enough to ever see the number of Task objects decreasing or limiting.

So it is the usual story, accusations of the .NET Framework having leaks, especially on these kind of essential object types that are used heavily in server-style apps that run for months, are forever highly exaggerated. Double-guessing the garbage collector is always tricky, you typically only gain confidence by in fact having your app running for months and never failing on OOM.