.NET 4 WCF memory issue

Question

I hit issue that my ASP.NET 4 MVC 2 + WCF application utilize lots of memory in Windows 2008 64-bit application during the load testing where it will use up almost all the available memory (8 GB) after few minutes run (we did have few worker process running).

After profiling using ANTS Memory Profiler it was showing few interesting outcome:

1. .NET managed memory increase from 15 MB to 40 MB, However this is attributed to caching mechanism that we did in the program. However .NET itself allocate allocate almost 180 MB Free Space, which is unexpected.
2. Unmanaged Memory size increases significantly until 120 MB after the load test just running about 3 minutes (although our application didn't explicitly use any P/Invoke or COM object. However we did use a few COM+ object which is disposed after used in the finally block).
3. Memory become fragmented.
4. Both item no 1 & 2 above causing the whole application to use about 350 MB just after the load test running a few minutes but if we didn't stop the test it will continue to grow further.

Based on item no 1 above, I tested some application to test whether the issue is due to our application or WCF. The test application just load XML data (about 300KB) to dataset in a multithreading application. When the logic is stored in EXE program, the application use only 200 KB (additional 120 KB from beginning with 40 KB for unused memory) managed memory from 24 MB private bytes after finish (which is acceptable); but when the logic is hosted in WCF, the application uses 66 MB managed memory (additional 61 MB from beginning with 64 MB free/unused managed memory). So it seems that WCF / ASP.NET is the one that causing the memory to increase a lot).

1. Why .NET allocate so much free space in the heap? Understood that the free space could be some Gen 0/Gen 1/Gen 2 that is GC-ed during memory snapshot process, but I don't think the application really use up that much memory.
2. is the behavior normal for WCF? If yes, any way to change the behavior so that it uses lesser memory?
3. How to find unmanaged memory leak especially that I didn't use the unmanaged code explicitly?

Appreciate your advise on the question above.

Thanks in advance,

Willy

Answer 1

WCF uses temporary buffers to process messages. What you perceive as a memory leak may be temporary buffers that haven't been collected yet.

To avoid creating new buffers all the time WCF uses BufferManager to reuse buffers, up to the limit specified by maxBufferPoolSize(link to the element here), which by default is 512KB. Any requests beyond this limit cause new buffers to be created that are never reused and have to be garbage collected.

Another option to check is maxBufferSize, which limits the maximum buffer size that can be returned by the BufferManager. Larger buffers are not pooled and have to be garbage collected. If you use large messages, you may be able to reduce the temporary buffers by increasing this property.

Try increasing maxBufferPoolSize to see if you can reduce memory usage. I would strongly advice though NOT to max it, because buffers from the pool are never released until the app-domain (ie the Application Pool) recycles. A period of high traffic could cause a lot of memory to be used and never released.

Answer 2

Regarding the "unmanaged memory leak" I had such a case sometime ago and after some digging it was an ADO.NET provider - fixed this with an updated version :-)

As for the rest - try putting this into the config:

<Configuration>
    <runtime>
        <gcServer enabled=“true“ />
    </runtime>
</Configuration>

Other than this there is nothing special about WCF IMHO... there could be a memory leak like in any .NET app (for example event handler and static handlers/objects can create memory leaks when not unsubsrcibed properly)...