Can I run out of virtual memory on linux?

Question

My application similar to hypotetical program:

for(;;) {
  for (i=0; i<1000; i++) {
    p[i] = malloc(random_number_between_1000_and_100000());
    p[i][0]=0;  // update
   }
  for (i=0; i<1000; i++) {
    free(p[i]);
  }
}

Has no memory leaks but on my system, the consumption of memory (top, column VSS) grows without limits (such as to 300% of available physical memory). Is this normal?

Updated - use the memory for a while and then free it. Is this a difference?

Answer 1

The behavior is normal. Quoting man 3 malloc:

BUGS

By default, Linux follows an optimistic memory allocation strategy. This means that when malloc() returns non-NULL there is no guarantee that the memory really is available. This is a really bad bug. In case it turns out that the system is out of memory, one or more processes will be killed by the infamous OOM killer. In case Linux is employed under circumstances where it would be less desirable to suddenly lose some randomly picked processes, and moreover the kernel version is sufficiently recent, one can switch off this overcommitting behavior using a command like:

       # echo 2 > /proc/sys/vm/overcommit_memory

See also the kernel Documentation directory, files vm/overcommit-accounting and sysctl/vm.txt.

You need to touch (read/write) the memory for the Linux kernel to actually reserve it.

Answer 2

Try to add

  sbrk(-1);

at end of each loop to see if it makes difference.

The free() only deallocates memory but it doesn't give it back to OS.