How to detect .NET WPF memory leak or GC long run?

Question

I have the next very strange situation and problem:

• .NET 4.0 application for diagram editing (WPF).

• Runs ok in my PC: 8GB RAM, 3.0GHz, i7 quad-core.

• While creating objects (mostly diagram nodes and connectors, plus all the undo/redo information) the TaskManager show, as expected, some memory usage "jumps" (up and down).

• These mem-usage "jumps" also remains executing AFTER user interaction ended. Maybe this is the GC cleaning/regorganizing memory?

• To see what is going on, I've used the Ants mem profiler, but somewhat it prevents those "jumps" to happen after user interaction.

PROBLEM: It Freezes/Hangs after seconds or minutes of usage in some slow/weak laptos/netbooks of my beta testers (under 2GHz of speed and under 2GB of RAM). I was thinking of a memory leak, but...

EDIT 1: Also, there is the case that the memory usage grows and grows until collapse (only in slow machines).

• In a Windows XP Mode machine (VM in Win 7) with only 512MB of RAM Assigned it works fine without mem-usage "jumps" after user interaction (no GC cleaning?!).

EDIT 2: The problem is worse when the system has some other heavy programs running (like Visual Studio, Office and Web pages open). Not even the first symbol of diagram can be created while the memory usage takes off like a rocket to space (hundreds of MBs consumed in seconds). Anyone with similar experiences? what were they strategies?

So, I really have a big trouble because I cannot reproduce the error, only see these strange behaviour (mem jumps), and the tool supposed to show me what is happening is hiding the problem (like the "observer's paradox").

Any ideas on what's happening and how to solve it?

EDIT 3: This screenshot of the Ants memory profiler shows that the huge consumption of ram (in crescendo) if from unmanaged resources.

But, what can be consuming so much memory, so fast??!!!

Answer 1

What you describe is all entirely normal behavior for a .NET program, there is no indication that there's anything wrong with your code.

By far the biggest issue is that TaskMgr.exe is just not a very good program to tell you what's happening in your process. It displays the "working set" for a process, a number that has very little to do with the amount of memory the process uses.

Working set is the amount of RAM that your process uses. Every process gets 2 gigabytes of virtual memory to use for code and data. Even on your virtual XP box with only 512 MB of RAM. All of those processes however have only a set amount of RAM to work with. On a lowly machine that can be as little as a gigabyte.

Clearly having multiple processes running, each with gigabytes of virtual memory with only a gigabyte of real memory takes some magic. That's provided by the operating system, Windows virtualizes the RAM. In other words, it creates the illusion for each process that it is running by its own on a machine with 2 gigabytes of RAM. This is done by a feature called paging, whenever a process needs to read or write memory, the operating system grabs a chunk of RAM to provide the physical memory.

Inevitably, it has to take away some RAM from another process so that it can be made available to yours. Whatever was previously in that chunk of RAM needs to be preserved. That's what the paging file does, it stores the content of RAM that was paged out.

Clearly this does not come for free, disks are pretty slow and paging is an expensive operation. That's why lowly machines perform poorly when you ask them to run several large programs. The real measure for this is also visible in TaskMgr.exe but you have to add it. View + Select Columns and tick "Page fault delta". Observe this number while your process runs. When you see it spike, you can expect your program to slow down a great deal and the displayed memory usage to change rapidly.

Addressing your observations:

creating objects ... the TaskManager show, as expected, some memory usage "jumps"

Yes, you are using RAM so the working set goes up.

These mem-usage "jumps" also remains executing AFTER user interaction ended

No slam dunk, but other processes get more time to execute, using RAM in turn and bumping some of yours out. Check the Page fault delta column.

I've used the Ants mem profiler, but somewhat it prevents those "jumps" to happen after user interaction.

Yes, memory profilers focus on real memory usage of your program, the virtual memory kind. They largely ignore working set, there isn't anything you can do about it and the number is meaningless because it really depends on what other processes are running.

there is the case that the memory usage grows and grows until collapse

That can be a side-effect of the garbage collector but that isn't typical. You are probably just seeing Windows trimming your working set, chucking out pages so you don't consume too many.

In a Windows XP Mode machine (VM in Win 7) with only 512MB of RAM Assigned it works fine

That's likely because you haven't installed any large programs on that WM that would compete for RAM. XP was also designed to work well on machines with very little memory, it is smooth on a machine with 256 MB. That's most definitely not the case for Vista/Win7, they were designed to take advantage of modern machine hardware. A feature like Aero is nice eye candy but very expensive.

The problem is worse when the system has some other heavy programs running

Yes, you are competing with those other processes needing lots of RAM.

Not even the first symbol of diagram can be created while the memory usage takes off like a rocket

Yes, you are seeing pages getting mapped back to RAM, getting reloaded from the paging file and the ngen-ed .ni.dll files. Rapidly increasing your working set again. You'll also see the Page fault delta number peaking.

So concluding, your WPF program just consumes a lot of memory and needs the horse power to operate well. That's not easy to fix, it takes a pretty drastic redesign to lower the resource requirements. So just put the system requirements on the box, it is entirely normal to do so.

Answer 2

This would suggest that you're likely creating a lot of "garbage" - basically, creating and letting many objects go out of scope quickly, but which take long enough to get into Gen1 or Gen2. This puts a large burden on the GC, which in turn can cause freezes and hangs on many systems.

To see what is going on, I've used the Ants mem profiler, but somewhat it prevents those "jumps" to happen after user interaction.

The reason this profiler (ANTS), specifically, could mask this behavior is that it forces a full GC every time you take a memory snapshot. This would make it look like there is no memory "leak" (as there isn't), but does not show the total memory pressure on the system.

A tool like PerfView could be used to investigate the GC's behavior during the runtime of the process. This would help you track the number of GCs that occur, and your application state at that point in time.