Fail to malloc big block memory after many malloc/free small blocks memory

Question

Here is the code.

First I try to malloc and free a big block memory, then I malloc many small blocks memory till it run out of memory, and I free ALL those small blocks.

After that, I try to malloc a big block memory.

#include <stdio.h>
#include <stdlib.h>
int main (int argc, char **argv)
{
    static const int K = 1024;
    static const int M = 1024 * K;
    static const int G = 1024 * M;

    static const int BIG_MALLOC_SIZE = 1 * G;
    static const int SMALL_MALLOC_SIZE = 3 * K;
    static const int SMALL_MALLOC_TIMES = 1 * M;

    void **small_malloc = (void **)malloc(SMALL_MALLOC_TIMES * sizeof(void *));

    void *big_malloc = malloc(BIG_MALLOC_SIZE);
    printf("big malloc first time %s\n", (big_malloc == NULL)? "failed" : "succeeded");
    free(big_malloc);

    for (int i = 0; i != SMALL_MALLOC_TIMES; ++i)
    {
        small_malloc[i] = malloc(SMALL_MALLOC_SIZE);
        if (small_malloc[i] == NULL)
        {
            printf("small malloc failed at %d\n", i);
            break;
        }
    }
    for (int i = 0; i != SMALL_MALLOC_TIMES && small_malloc[i] != NULL; ++i)
    {
        free(small_malloc[i]);
    }

    big_malloc = malloc(BIG_MALLOC_SIZE);
    printf("big malloc second time %s\n", (big_malloc == NULL)? "failed" : "succeeded");
    free(big_malloc);

    return 0;
}

Here is the result:

big malloc first time succeeded
small malloc failed at 684912
big malloc second time failed

It looks like there are memory fragments.

I know memory fragmentation happens when there are many small empty space in memory but there is no big enough empty space for big size malloc.

But I've already free EVERYTHING I malloc, the memory should be empty.

Why I can't malloc big block at the second time?

I use Visual Studio 2010 on Windows 7, I build 32-bits program.

Answer 1

The answer, sadly, is still fragmentation.

Your initial large allocation ends up tracked by one allocation block; however when you start allocating large numbers of 3k blocks of memory your heap gets sliced into chunks.

Even when you free the memory, small pieces of the block remain allocated within the process's address space. You can use a tool like Sysinternals VMMap to see these allocations visually.

It looks like 16M blocks are used by the allocator, and once these blocks are freed up they never get returned to the free pool (i.e. the blocks remain allocated).

As a result you don't have enough contiguous memory to allocate the 1GB block the second time.