Why does malloc() or new never return NULL? [duplicate]

Question

I'm writing an application which needs a lot of memory for caching purposes as I described he here. Now I'm playing around with some malloc / new constructions to figure out how I could realise it. I made a strange observation:

#include <stdio.h>
#include <stdlib.h>

int main(void) {
  while(1) {
    char *foo = (char*)malloc(1024);// new char[1024];
    if(foo == NULL) { 
      printf("Couldn't alloc\n");
      fflush(stdout);
      return 0;
    }
  }
  return 0;
}

Why does that printf never be reached? If my system runs out of memory, malloc is said to return NULL, as it is explained here. But I always receive SIGKILL (I'm using linux...).

Answer 1

Linux, by default, usually uses an opportunistic memory allocation scheme, meaning the kernel will give you a valid address that won't be allocated until first use.

See:

• SIGKILL while allocating memory
• C Program on Linux to exhaust memory

According to those responses you can turn this feature off using echo 2 > /proc/sys/vm/overcommit_memory.

From what I can tell, this is done under the assumption that you wont necessarily use all the memory that you allocate. I can't say that I personally ever allocate space that I don't touch at least once, so I'd be curious to know how this affects real life performance...

Regarding the SIGKILL failure, every malloc you call is still allocating some memory for each call. Eventually you will likely fill your memory with malloc overhead and thus evoke the fury of the out of memory kill feature. Whether this alone is the issue or if perhaps the overcommit policy still allocates some fraction of the requested space is a good question.

Answer 2

Usually, Linux will allocate as much (virtual) memory as you request, and only allocate physical memory for it when it's needed. If the system runs out of physical memory, then it starts killing processes to free some.

This means that malloc will succeed unless the request is ludicrous, but your process (or some other) is likely to get killed as the memory is used.

For more details, see the manpage for malloc, and its references:

By default, Linux follows an optimistic memory allocation strategy. This means that when malloc() returns non-NULL there is no guarantee that the memory really is available. In case it turns out that the system is out of memory, one or more processes will be killed by the OOM killer. For more information, see the description of /proc/sys/vm/overcommit_memory and /proc/sys/vm/oom_adj in proc(5), and the kernel source file Documentation/vm/overcommit-accounting.

(And of course new won't return null anyway unless you use the no-throwing version).

Answer 3

malloc returns NULL if requested allocation cannot be fulfilled. But maybe you should try allocating tons of space from heap.

See here.

On linux, the only way to get an error when calling malloc() is to disable memory-overcommiting. On regular linux systems, this is the only way for malloc() to return NULL. If an icecast process reaches that point, it's screwed anyway it won't be able to do anything meaningful: any source reads will fail (refbuf alloc), any log print will also fail (printf uses malloc too), so it might as well give up and call abort().