what does "malloc_trim(0)" really mean?

Question

The manual page told me so much and through it I know lots of the background knowledge of memory management of "glibc".

But I still get confused. does "malloc_trim(0)"(note zero as the parameter) mean (1.)all the memory in the "heap" section will be returned to OS ? Or (2.)just all "unused" memory of the top most region of the heap will be returned to OS ?

If the answer is (1.), what if the still used memory in the heap? if the heap has used momery at places somewhere, will them be eliminated, or the function wouldn't execute successfully?

While if the answer is (2.), what about those "holes" at places rather than top in the heap？　they're unused memory anymore, but the top most region of the heap is still used, will this calling work efficiently?

Thanks.

Answer 1

Man page of malloc_trim was committed here: https://github.com/mkerrisk/man-pages/blob/master/man3/malloc_trim.3 and as I understand, it was written by man-pages project maintainer, kerrisk in 2012 from scratch: https://github.com/mkerrisk/man-pages/commit/a15b0e60b297e29c825b7417582a33e6ca26bf65

As I can grep the glibc's git, there are no man pages in the glibc, and no commit to malloc_trim manpage to document this patch. The best and the only documentation of glibc malloc is its source code: https://sourceware.org/git/?p=glibc.git;a=blob;f=malloc/malloc.c There are malloc_trim comments from malloc/malloc.c:

Additional functions:
 malloc_trim(size_t pad);
 609 /*
 610   malloc_trim(size_t pad);
 611 
 612   If possible, gives memory back to the system (via negative
 613   arguments to sbrk) if there is unused memory at the `high' end of
 614   the malloc pool. You can call this after freeing large blocks of
 615   memory to potentially reduce the system-level memory requirements
 616   of a program. However, it cannot guarantee to reduce memory. Under
 617   some allocation patterns, some large free blocks of memory will be
 618   locked between two used chunks, so they cannot be given back to
 619   the system.
 620 
 621   The `pad' argument to malloc_trim represents the amount of free
 622   trailing space to leave untrimmed. If this argument is zero,
 623   only the minimum amount of memory to maintain internal data
 624   structures will be left (one page or less). Non-zero arguments
 625   can be supplied to maintain enough trailing space to service
 626   future expected allocations without having to re-obtain memory
 627   from the system.
 628 
 629   Malloc_trim returns 1 if it actually released any memory, else 0.
 630   On systems that do not support "negative sbrks", it will always
 631   return 0.
 632 */
 633 int      __malloc_trim(size_t);
 634 

Freeing from the middle of the chunk is not documented as text in malloc/malloc.c and not documented in man-pages project. Man page from 2012 may be the first man page of the function, written not by authors of glibc. Info page of glibc only mentions M_TRIM_THRESHOLD of 128 KB: https://www.gnu.org/software/libc/manual/html_node/Malloc-Tunable-Parameters.html#Malloc-Tunable-Parameters and don't list malloc_trim function https://www.gnu.org/software/libc/manual/html_node/Summary-of-Malloc.html#Summary-of-Malloc (and it also don't document memusage/memusagestat/libmemusage.so).

In december 2007 there was commit https://sourceware.org/git/?p=glibc.git;a=commit;f=malloc/malloc.c;h=68631c8eb92ff38d9da1ae34f6aa048539b199cc by Ulrich Drepper (it is part of glibc 2.9 and newer) which changed mtrim implementation (but it didn't change any documentation or man page as there are no man pages in glibc):

• malloc/malloc.c (public_mTRIm): Iterate over all arenas and call

mTRIm for all of them. (mTRIm): Additionally iterate over all free blocks and use madvise to free memory for all those blocks which contain at least one memory page.

Unused parts of chunks (anywhere, including chunks in the middle), aligned on page size and having size more than page may be marked as MADV_DONTNEED https://sourceware.org/git/?p=glibc.git;a=blobdiff;f=malloc/malloc.c;h=c54c203cbf1f024e72493546221305b4fd5729b7;hp=1e716089a2b976d120c304ad75dd95c63737ad75;hb=68631c8eb92ff38d9da1ae34f6aa048539b199cc;hpb=52386be756e113f20502f181d780aecc38cbb66a

       INTERNAL_SIZE_T size = chunksize (p);

       if (size > psm1 + sizeof (struct malloc_chunk))
         {
           /* See whether the chunk contains at least one unused page.  */
           char *paligned_mem = (char *) (((uintptr_t) p
                                           + sizeof (struct malloc_chunk)
                                           + psm1) & ~psm1);

           assert ((char *) chunk2mem (p) + 4 * SIZE_SZ <= paligned_mem);
           assert ((char *) p + size > paligned_mem);

           /* This is the size we could potentially free.  */
           size -= paligned_mem - (char *) p;

           if (size > psm1)
               madvise (paligned_mem, size & ~psm1, MADV_DONTNEED);
        }

This is one of total two usages of madvise with MADV_DONTNEED in glibc now, one for top part of heaps (shrink_heap) and other is marking of any chunk (mtrim): http://code.metager.de/source/search?q=MADV_DONTNEED&path=%2Fgnu%2Fglibc%2Fmalloc%2F&project=gnu

 H A D  arena.c 643 __madvise ((char *) h + new_size, diff, MADV_DONTNEED);
 H A D  malloc.c    4535 __madvise (paligned_mem, size & ~psm1, MADV_DONTNEED);

We can test the malloc_trim with this simple C program (test_malloc_trim.c) and strace/ltrace:

#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
#include <malloc.h>

int main()
{
    int *m1,*m2,*m3,*m4;
    printf("%s\n","Test started");
    m1=(int*)malloc(20000);
    m2=(int*)malloc(40000);
    m3=(int*)malloc(80000);
    m4=(int*)malloc(10000);
    // check that all arrays are allocated on the heap and not with mmap
    printf("1:%p 2:%p 3:%p 4:%p\n", m1, m2, m3, m4);
    // free 40000 bytes in the middle
    free(m2);
    // call trim (same result with 2000 or 2000000 argument)
    malloc_trim(0);
    // call some syscall to find this point in the strace output
    sleep(1);
    free(m1);
    free(m3);
    free(m4);
    // malloc_stats(); malloc_info(0, stdout);
    return 0;
}

gcc test_malloc_trim.c -o test_malloc_trim, strace ./test_malloc_trim

write(1, "Test started\n", 13Test started
)          = 13
brk(0)                                  = 0xcca000
brk(0xcef000)                           = 0xcef000
write(1, "1:0xcca010 2:0xccee40 3:0xcd8a90"..., 441:0xcca010 2:0xccee40 3:0xcd8a90 4:0xcec320
) = 44
madvise(0xccf000, 36864, MADV_DONTNEED) = 0
...
nanosleep({1, 0}, 0x7ffffafbfff0)       = 0
brk(0xceb000)                           = 0xceb000

So, there was madvise with MADV_DONTNEED for 9 pages after malloc_trim(0) call, when there was hole of 40008 bytes in the middle of the heap.

Answer 2

The man page for malloc_trim says it releases free memory, so if there is allocated memory in the heap, it won't release the whole heap. The parameter is there if you know you're still going to need a certain amount of memory, so freeing more than that would cause glibc to have to do unnecessary work later.

As for holes, this is a standard problem with memory management and returning memory to the OS. The primary low-level heap management available to the program is brk and sbrk, and all they can do is extend or shrink the heap area by changing the top. So there's no way for them to return holes to the operating system; once the program has called sbrk to allocate more heap, that space can only be returned if the top of that space is free and can be handed back.

Note that there are other, more complex ways to allocate memory (with anonymous mmap, for example), which may have different constraints than sbrk-based allocation.