Ubuntu System Monitor and valgrind to discover memory leaks in C++ applications

Question

I'm writing an application in C++ which uses some external open source libraries. I tried to look at the Ubuntu System Monitor to have information about how my process uses resources, and I noticed that resident memory continues to increase to very large values (over 100MiB). This application should run in an embedded device, so I have to be careful.

I started to think there should be a (some) memory leak(s), so I'm using valgrind. Unfortunately it seems valgrind is not reporting significant memory leaks, only some minor issues in the libraries I'm using, nothing more.

So, do I have to conclude that my algorithm really uses that much memory? It seems very strange to me... Or maybe I'm misunderstanding the meaning of the columns of the System Monitor? Can someone clarify the meaning of "Virtual Memory", "Resident Memory", "Writable Memory" and "Memory" in the System Monitor when related to software profiling? Should I expect those values to immediately represent how much memory my process is taking in RAM?

In the past I've used tools that were able to tell me where I was using memory, like Apple Profiling Tools. Is there anything similar I can use in Linux as well?

Thanks!

Answer 1

Another tool you can try is the /lib/libmemusage.so library:

$ LD_PRELOAD=/lib/libmemusage.so vim 

Memory usage summary: heap total: 4643025, heap peak: 997580, stack peak: 26160
         total calls   total memory   failed calls
 malloc|      42346        4528378              0
realloc|         52           7988              0  (nomove:26, dec:0, free:0)
 calloc|         34         106659              0
   free|      28622        3720100
Histogram for block sizes:
    0-15          14226  33% ==================================================
   16-31           8618  20% ==============================
   32-47           1433   3% =====
   48-63           4174   9% ==============
   64-79           4736  11% ================
   80-95            313  <1% =
...

(I quit vim immediately after startup.)

Maybe the histogram of block sizes will give you enough information to tell where leaks may be happening.

valgrind is very configurable; --leak-check=full --show-reachable=yes might be a good starting point, if you haven't tried it yet.

---

"Virtual Memory", "Resident Memory", "Writable Memory" and "Memory"

Virtual memory is the address space that your application has allocated. If you run malloc(1024*1024*100);, the malloc(3) library function will request 100 megabytes of storage from the operating system (or handle it out of the free lists). The 100 megabytes will be allocated with mmap(..., MAP_ANONYMOUS), which won't actually allocate any memory. (See the rant at the end of the malloc(3) page for details.) The OS will provide memory the first time each page is written.

Virtual memory accounts for all the libraries and executable objects that are mapped into your process, as well as your stack space.

Resident memory is the amount of memory that is actually in RAM. You might link against the entire 1.5 megabyte C library, but only use the 100k (wild guess) of the library required to support the Standard IO interface. The rest of the library will be demand paged in from disk when it is needed. Or, if your system is under memory pressure and some less-recently-used data is paged out to swap, it will no longer count against Resident memory.

Writable memory is the amount of address space that your process has allocated with write privileges. (Check the output of pmap(1) command: pmap $$ for the shell, for example, to see which pages are mapped to which files, anonymous space, the stack, and the privileges on those pages.) This is a reasonable indication of how much swap space the program might require in a worst-case swapping scenario, when everything must be paged to disk, or how much memory the process is using for itself.

Because there are probably 50--100 processes on your system at a time, and almost all of them are linked against the standard C library, all the processes get to share the read-only memory mappings for the library. (They also get to share all the copy-on-write private writable mappings for any files opened with mmap(..., MAP_PRIVATE|PROT_WRITE), until the process writes to the memory.) The top(1) tool will report the amount of memory that can be shared among processes in the SHR column. (Note that the memory might not be shared, but some of it (libc) definitely is shared.)

Memory is very vague. I don't know what it means.