Memory usage doesn't decrease when free() used [duplicate]

Question

I am writing OS independent lockless queue, so far it works great, but there is small problem with memory managment. I am not sure if its gcc problem or mine. Problem: Memory increases when element is added to list, but when element is removed from list (free(elementPointer);) memory usage don't change.

BUT when I use pthreads, N producers and M consumers (1<N<20, 1<M<20) memory usage is about ~10mb all the time (when trying to add and remove ~10kk elements), so looks like free is working.

And funny thing is that in VS 2010 (same code, no threads) free works fine, memory is released (watched task manager).

I made test, added 1kk elements, after adding all, removed one by one all elements (no threads).

Linux - 0.08 seconds

Windows ~57 seconds

Linux(without free) - 0.07 seconds

Windows(without free) - 0.9 seconds

So, the question is, why memory isn't freed in Linux C when no threads are used ? I can post code if necessary .

GCC version: 4.4.3

Answer 1

On many operating systems, free() doesn't make the memory available for the OS again, but "only" for new calls to malloc(). This is why you don't see the memory usage go down externally, but when you increase the number of new allocations by threading, the memory is re-used so total usage doesn't go through the roof.

Answer 2

To oversimplify things, there are two memory managers at work in dynamic memory allocation: the OS Memory Manager, and the Process Memory Manager (which there can be more than one of). The OS Memory Manager allocates "large chunks" of memory to individual Process Memory Managers. Each Process Memory Manager keeps track of the allocated segments, as well as the "free'd segments". The Process Memory Manager does not return the free'd segments to the OS Memory Manager because it is more efficient to hold on to it, in case it needs to allocate more memory later.

Answer 3

Malloc doesn't have to return memory to the operating system. Most malloc implementations on unix-like systems don't do it. Especially for smaller object sizes.

This is done for performance reasons.

I just noticed that this is potentially unclear. By "malloc" I mean the whole subsystem related to the malloc function - malloc, free, realloc, calloc and whatever special functions your libc might implement.