Find memory leaks with WinDbg when lots of objects are present in Gen2

Question

I've got a memory problem in my .NET application where my app starts off consuming about 1GB in the Gen2 heap after everything is initialized and loaded. It slowly over time (4-5 hours) ends up consuming 4GB in the Gen2 heap. I've used WinDbg to analyze things that I see that some of my object types (and associated memory usage) are increasing.

All of the objects that growing in instances (and mem usage) are referenced by the same parent object type. This parent object type has about 3900 instances - this never changes. Somehow, I'm adding child objects to some of these parent instances, but I don't have a good way to see which of the 3900 instances are being added to.

!DumpHeap -mt will show me all my parent types, but the sizes are all the same because it doesn't count children.

!ObjSize will count the size of the children too, but will only take one object at a time for an argument (or all objects of all types - not just my parent type - which is way too many objects)

Looking at the child objects and tracing them back to the parent isn't helpful either because there are a couple million of these types and I don't see a way to do some kind of aggregate trace.

Tools like CLRProfiler and ANTS slow down my app too much (ANTS less so) to make the problem happen in any reasonable amount of time.

I have tried running my application with a small subset of data that it normally runs on in order to make debugging easier, but I don't get memory issues here. I think there are some edge cases in my entire data set that cause strange things, but I don't know what those edge cases are in order to isolate them into a subset of my entire data set.

Have read widely on this and can't see anyone suggesting what to do when are LOTS of objects in Gen2 that are supposed to be there and a small amount of objects of the same type that keep increasing.

Any tips would be most appreciated.

Answer 1

Interesting puzzle. As you know, objects get promoted to gen2 when they've survived collections because they're referenced by something that lives a long time. You don't say what kind of application this is - asp.net, WPF, winforms, etc. so we'll have to make some guesses.

One strategy you could try is logging. You say there are 3900 instances of a 'parent object' that something is getting added to - can you instrument the method on the parent object that accepts the new objects? Maybe by logging those additions you can get an idea of where they're coming from.

Answer 2

I assume you're using SOS for this. A better option would be to use PSSCOR2 or PSSCOR4 (depending on which version of the runtime you're on). The PSSCOR version of !dumpheap has an additional delta column, which might help you detect which instances are growing over time.