Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

new[] doesn't decrease available memory until populated

This is in C++ on CentOS 64bit using G++ 4.1.2.

We're writing a test application to load up the memory usage on a system by n Gigabytes. The idea being that the overall system load gets monitored through SNMP etc. So this is just a way of exercising the monitoring.

What we've seen however is that simply doing:

char* p = new char[1000000000];

doesn't affect the memory used as shown in either top or free -m

The memory allocation only seems to become "real" once the memory is written to:

memcpy(p, 'a', 1000000000);   //shows an increase in mem usage of 1GB

But we have to write to all of the memory, simply writing to the first element does not show an increase in the used memory:

p[0] = 'a';    //does not show an increase of 1GB.

Is this normal, has the memory actually been allocated fully? I'm not sure if it's the tools we are using (top and free -m) that are displaying incorrect values or whether there is something clever going on in the compiler or in the runtime and/or kernel.

This behavior is seen even in a debug build with optimizations turned off.

It was my understanding that a new[] allocated the memory immediately. Does the C++ runtime delay this actual allocation until later on when it is accessed. In that case can an out of memory exception be deferred until well after the actual allocation of the memory until the memory is accessed?

As it is it is not a problem for us, but it would be nice to know why this is occurring the way it is!

Cheers!

Edit:

I don't want to know about how we should be using Vectors, this isn't OO / C++ / the current way of doing things etc etc. I just want to know why this is happening the way it is, rather than have suggestions for alternative ways of trying it.

like image 706
fwg Avatar asked Mar 17 '11 15:03

fwg


People also ask

Does C++ allocate memory automatically?

In C language, we use the malloc() or calloc() functions to allocate the memory dynamically at run time, and C++ also supports these functions. But, in C++, allocation and deallocation are done manually.

Should you ever use new in C++?

The new operator should only be used if the data object should remain in memory until delete is called. Otherwise if the new operator is not used, the object is automatically destroyed when it goes out of scope.

What is the difference between new and malloc in C++?

malloc(): It is a C library function that can also be used in C++, while the “new” operator is specific for C++ only. Both malloc() and new are used to allocate the memory dynamically in heap. But “new” does call the constructor of a class whereas “malloc()” does not.

How do I free up my CPP memory?

The free() function in C++ deallocates a block of memory previously allocated using calloc, malloc or realloc functions, making it available for further allocations. The free() function does not change the value of the pointer, that is it still points to the same memory location.


1 Answers

When your library allocates memory from the OS, the OS will just reserve an address range in the process's virtual address space. There's no reason for the OS to actually provide this memory until you use it - as you demonstrated.

If you look at e.g. /proc/self/maps you'll see the address range. If you look at top's memory use you won't see it - you're not using it yet.

like image 149
Erik Avatar answered Oct 24 '22 20:10

Erik