Virtual Size causing program to run out of memory

Question

A server application I'm working on built using C++ on Windows runs out of memory when Virtual Size reaches somewhere around 2GB (32-bit application, with large address aware enabled). I notice, however, that Private Bytes is significantly smaller. The current statistics are:

Virtual Size: 2.6GB Private Bytes: 1.6GB

The difference in these two numbers is 1GB. So my questions are:

1. What does this 1GB difference represent?
2. Is my application running out of memory due to Virtual Size or Private Bytes?

I've also run my application through the VMMap utility and I notice that "Private Data" is usually an order of magnitude higher than the committed size. In other words, total size for Private Data might be 200MB, but the committed size is only 20MB. I'm not really sure what private data is, but based on my research so far it seems to indicate it's just part of the heap.

EDIT:

I've looked for memory leaks using Purify but I've not really found anything useful. Memory leaks in the form of memory without pointers doesn't seem to be the problem, but memory leaks in terms of memory being held onto for too long might be an issue, i haven't looked into that yet. However, the key is understanding why Virtual Size is causing the out of memory issues. Question #1 is the most important for me to understand this.

Answer 1

This is going to require a little explanation, so stick with me here.

First off, this topic is a confusing quagmire of conflicting terms, so please throw away all notions you have of "virtual memory" being anything to do with disks.

• Physical memory is memory stored on a physical device. This usually refers to system RAM, but can also be disk buffers, NIC buffers, graphics card VRAM, etc.
• Virtual memory is a set of physical address ranges mapped into user-mode (virtual) address ranges so that memory can be accessed in a safe and compartmentalised way.

A quick note on why we do this: if we gave processes direct memory addresses, we could only (feasibly) have a single memory store. This is inconvenient and bad for performance. When a virtual address translates to a physical address outside the range of the system memory (RAM), the processor issues a page fault. This signals an interrupt handler in the OS, which can then delegate the memory access operation to a different device. Useful!

On a 32-bit Windows system, the maximum amount of virtual memory that a process can address at any one point in time is 2GB. This can be increased to 3GB using AWE, or 4GB with /4GT and AWE. This does not mean that a process can only ever allocate 2GB (or 3GB / 4GB depending on the previously discussed settings) of memory. It just means that it cannot have concurrent access to more than that.

For example, if you open a memory-mapped file that is 1GB in size, your virtual memory usage will increase by 1GB. You're not touching the RAM, nor the disk, but you've allocated a block of virtual address space to your process. If you then want to allocate 1.1GB of RAM at the same time as having this memory mapped file available, you cannot. You must first unmap the file from your process. Remember that memory can still remain allocated and full of data, but not actually mapped into your process. If you have 8GB of RAM on your machine, you could fill 6GB of it up with data and map 2GB of it into your process. When you need to use a different section of that memory, you must unmap the existing block and map the other section.

So, onto the difference you're seeing:

1. Private bytes tells you how many bytes of virtual device memory your process has mapped, excluding virtual memory shared with other processes (e.g. mapped files, global heap, etc).

2. Working set tells you how many bytes of physical memory you are actively using. This includes physical memory, device buffers, and mapped files. It's a pretty strange figure, since it equates to touched physical memory + mapped virtual non-system memory. In general, you should completely ignore this figure. It's practically useless for debugging memory leaks.

3. Virtual bytes is the total amount of virtual memory you have mapped.

The difference is that you've mapped shared virtual memory, for example a bunch of DLL files or a block of global heap, into your process. The difference indicates that the total size of these shared mappings is roughly 1GB.

Keep in mind that none of this has anything to do with swapping, which is the transfer of system memory pages to disk (the so-called "pagefile") to increase the availability of fast system resources (RAM). The naming of that file has caused no end of confusion in this area of Windows, and I'll be elated when Microsoft finally decide to call it "swap" rather than "virtual memory" or "page file".