Why malloc doesn't allocate memory until I hit a certain threshold?

Question

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>

int main(int argc, char *argv[])
{
        size_t sz = atol(argv[1]);
        char *arr = malloc(sz);

        sleep(10);
}

I compiled this code and tried to run it, used pmap to see the memory mapping of the program.

When I use some big number like 1024000, I get a mapping like this:

3901:   ./alloc_program 1024000
0000560192f43000      4K r---- alloc_program
0000560192f44000      4K r-x-- alloc_program
0000560192f45000      4K r---- alloc_program
0000560192f46000      4K r---- alloc_program
0000560192f47000      4K rw--- alloc_program
0000560192fac000    132K rw---   [ anon ]
00007f75b69e9000   1004K rw---   [ anon ]     <---- I believe this is the allocated memory
00007f75b6ae4000    148K r---- libc-2.31.so
00007f75b6b09000   1504K r-x-- libc-2.31.so
00007f75b6c81000    296K r---- libc-2.31.so
00007f75b6ccb000      4K ----- libc-2.31.so
00007f75b6ccc000     12K r---- libc-2.31.so
00007f75b6ccf000     12K rw--- libc-2.31.so
00007f75b6cd2000     24K rw---   [ anon ]
00007f75b6ce7000      4K r---- ld-2.31.so
00007f75b6ce8000    140K r-x-- ld-2.31.so
00007f75b6d0b000     32K r---- ld-2.31.so
00007f75b6d14000      4K r---- ld-2.31.so
00007f75b6d15000      4K rw--- ld-2.31.so
00007f75b6d16000      4K rw---   [ anon ]
00007ffe2b26e000    132K rw---   [ stack ]
00007ffe2b318000     12K r----   [ anon ]
00007ffe2b31b000      4K r-x--   [ anon ]
ffffffffff600000      4K --x--   [ anon ]
 total             3496K

I suppose that the marked line is the memory allocated by malloc (maybe I'm wrong). But when I use some small number like 10240, I don't see that anything is allocated:

3879:   ./alloc_program 10240
000055e428e26000      4K r---- alloc_program
000055e428e27000      4K r-x-- alloc_program
000055e428e28000      4K r---- alloc_program
000055e428e29000      4K r---- alloc_program
000055e428e2a000      4K rw--- alloc_program
000055e42a257000    132K rw---   [ anon ]
00007f102332c000    148K r---- libc-2.31.so
00007f1023351000   1504K r-x-- libc-2.31.so
00007f10234c9000    296K r---- libc-2.31.so
00007f1023513000      4K ----- libc-2.31.so
00007f1023514000     12K r---- libc-2.31.so
00007f1023517000     12K rw--- libc-2.31.so
00007f102351a000     24K rw---   [ anon ]
00007f102352f000      4K r---- ld-2.31.so
00007f1023530000    140K r-x-- ld-2.31.so
00007f1023553000     32K r---- ld-2.31.so
00007f102355c000      4K r---- ld-2.31.so
00007f102355d000      4K rw--- ld-2.31.so
00007f102355e000      4K rw---   [ anon ]
00007fff1d513000    132K rw---   [ stack ]
00007fff1d570000     12K r----   [ anon ]
00007fff1d573000      4K r-x--   [ anon ]
ffffffffff600000      4K --x--   [ anon ]
 total             2492K

1 - Why it doesn't allocate when the memory size is relatively small?

2 - Why the allocated memory size is not exactly the same? In the first run, it shows that the size is 1004KB while I've only allocated 1000KB.

Answer 1

1 - Why it doesn't allocate when the memory size is relatively small?

The task of the function malloc is to provide the application with memory, whenever it asks for it. Theoretically, malloc could, as you suggest, just forward all requests for memory allocations to the operating system's kernel, so that it only acts as a wrapper for the kernel's memory allocator. However, this has the following disadvantages:

1. The kernel only provides large amounts of memory at once, at least one page of memory, which is, depending on the configuration of the operating system, normally at least 4096 bytes. Therefore, if an application asked for only 10 bytes of memory, a lot of memory would be wasted.
2. System calls are expensive in terms of CPU performance.

For these reasons, it is more efficient for malloc to not forward memory allocation requests directly to the kernel, but to rather act as an intermediary between the application's memory allocation requests and the kernel. It requests memory in larger amounts from the kernel, so that it can satisfy many smaller memory allocation requests from the application.

Therefore, only when asking for a large amount of memory at once, will malloc forward that memory allocation request to the kernel.

---

2 - Why the allocated memory size is not exactly the same? In the first run, it shows that the size is 1004KB while I've only allocated 1000KB.

The malloc allocator must keep track of all the memory allocations it granted to the application and also keep track of all the memory allocations that it has been granted by the kernel. To store this information, it requires a bit of addititional memory space. This additional space is called "overhead".

Answer 2

What you're seeing in the pmap output is almost certainly the addition needed to the malloc arena to satisfy larger requests, not any single request.

The arena is the pool of memory from which allocations are handed out and there's a good chance this starts at a certain size and is only expanded on demand.

For example, if the initial arena is 1000K, any allocation that does not exhaust that will have no need to get extra arena space. If you do exhaust that space, the process will try to request more arena from the underlying environment so it can satisfy the extra demand.

---

As to why the size isn't what you requested, there are (at least) two possible reasons. First, the arena isn't just the memory allocated for your purposes, it also holds control information so that the memory can be properly managed (sizes, checksums, pointers, free list and so on).

Secondly, malloc may over-allocate on the expectation that this won't be the last request that exhausts the current arena. Some memory allocation strategies go so far as to double the current arena size when requesting more, in order to amortise the cost of doing so.