Why the "View Heap" result does not match with 'Process Memory Usage' in Visual Studio

Question

I am trying to use Visual Studio to track memory usage in my app. In the 'Diagnostic Tools' window, it shows my app is using 423 MB. Thank I go to 'Memory Usage' and 'ViewHeap', when I click on the snapshot, I get a table of the size of my objects.

But when I add those number up:

= 3317228 
+  403764
+  354832
+  264984
+  244836
+  195748
+  144032
+   28840
+   16452
+   13920
+   13888
+    3428
+    2100
+      20
= 5004072
= 4.77 MB

My question is why this number 4.77MB does not match with the 423MB I see on the "Memory" Chart. I expect to see the table on the left a breakdown of where 423 MB went. Please tell me what am I missing?

Answer 1

Why does the View Heap size not match the memory chart size?

There are dozens of potential reasons for this, including JITter, Debug Tools, Debug Symbols, Just My Code, Garbage Collection et al. We'll go through two of the big ones.

Just My Code

The Just My Code feature of Visual Studio tends to hide allocations, exceptions, breakpoints, and any other non-code meta-data from the user, that was not loaded from a .PDB file or an open project. See MSDN Just My Code for details.

Debugging Symbols and Tools

When debugging any project in Visual Studio, the Visual Studio Debugger runs and allocates extra memory to allow for breakpoints, exception catching, and other features. For a true diagnostic tools capture, you should use the Alt+F2 option, or Debug > Start Diagnostic Tools Without Debugging.... You'll also want to switch to Release mode for this portion. This step alone cut the memory the graph showed (for me) from 21.5MiB to 5.5MiB, indicating that the Debugging Symbols and Debugging Tools are a substantial factor. Remember, in order for Visual Studio to be able to catch exceptions, breakpoints and other data, it must attach itself to your process, and to all objects within your process.

So, how do we make these numbers match?

You really shouldn't worry about the numbers matching. The purpose of the Memory Graph and the View Heap chart is to allow you to see spikes and odd memory fluctuations, that could indicate programme incorrectness. You should be looking for those, rather than focusing on the difference between the two values.

That said, there are some steps you can take to get accurate results.

Truly Matching the Numbers

If you truly want to match them, I don't think it can be done in the manner you wish. You can, however, get closer. The first step is to Start Diagnostic Tools Without Debugging..., then select Memory Usage. Once selected, click the Settings Gear next to it, and make sure Profiler Type is Mixed (Managed and Native). Then, click Start and take some snapshots so you can examine the memory use. Once done so, stop your debugging and examine your memory.

To examine your memory, click the top-left blue number in the snapshot box for the snapshot you wish to examine. On this page, click the Grid Icon on the top-right and deselect both Just My Code and Collapse Small Objects. Switch to the Native Heap tab and do the same, deselecting Just My Code and then select Include Freed Allocations.

You should find that this alone brings your error well closer to the actual value. (The actual value being the Private Bytes and the error being the Heap Size) With the application I tested it on, it brought the total (from both heaps) to about 1.0265MiB, which was about the same as the allocation indicated by Task Manager when I ran the programme outside of Visual Studio (this actual value was 1.1211MiB, but with numbers this small that margin of error is expected).

What does Include Freed Allocations mean? Essentially, when the GC clears memory, that memory is not immediately removed from the space of the application. Instead it is freed for use by other objects, but can still remain with the application. Garbage Collection is a complicated topic, and well beyond the scope of this question and answer.

Additional Notes

Memory allocation, usage and measurement is a very complex topic. Unfortunately, there are not many 100% fool-proof ways to handle situations like this, and generally the more fool-proof and accurate the solution is, the more complex, slow and difficult to use it is.

References

MSDN Just My Code: https://msdn.microsoft.com/en-us/library/dn457346.aspx#BKMK__NET_Framework_Just_My_Code

MSDN Garbage Collection: https://msdn.microsoft.com/en-us/library/0xy59wtx%28v=vs.110%29.aspx

The rest of this answer is based on my own experimentation and trial and error, and is subject to potential inaccuracies which may result of different environments. The steps presented herein may not work for all developers, and were performed with Visual Studio 2015 RC version 14.0.22823.1 D14REL.

Answer 2

The graph shows Private Bytes for the entire process. This includes the Managed heap, native heap, stacks, etc... See this answer for more on types of memory metrics: What is private bytes, virtual bytes, working set?

The Heap View table only shows types active (not able to be garbage collected) on the Managed heap at the time the snapshot was taken. To see types on both the native and managed heaps, switch to mixed-mode debugging. The Heap View (and the number in snapshot table) are a subset of the process memory shown in the graph.

The debugger-integrated tool works best for trying to find the cause of unexpected growth in memory, or tracking down objects that should have been garbage collected but have references still keeping them alive.

Here's a blog post I wrote (I work for MSFT) on the Memory tool explaining how to track down objects with outdated references: https://web.archive.org/web/20150905153352/http://blogs.msdn.com/b/visualstudioalm/archive/2015/04/29/diagnosing-event-handler-leaks-with-the-memory-usage-tool-in-visual-studio-2015.aspx