Maximum memory that can be allocated dynamically and at compile time in c++

Question

I am playing around to understand how much memory can be allocated. Initially I thought that the maximum memory which can be allocated is equal to Physical memory (RAM). I checked my RAM on Ubuntu 12.04 by running the command as shown below:

~$ free -b
             total       used       free     shared    buffers     cached
Mem:    3170848768 2526740480  644108288          0  265547776 1360060416
-/+ buffers/cache:  901132288 2269716480
Swap:   2428497920          0 2428497920

As shown above,total physical memory is 3Gig (3170848768 bytes) out of which only 644108288 bytes is free, so I assumed I can at max allocate only this much memory. I tested it by writing the small program with only two lines below:

char * p1 = new char[644108290] ;
delete p1;

Since code ran perfectly , it means it allocated memory successfully. Also I tried to allocate the memory greater than the available physical free memory still it did not throw any error. Then per question

maximum memory which malloc can allocate

I thought it must be using the virtual memory.So I tested the code for free swap memory and it also worked.

char * p1 = new char[2428497920] ;
delete p1;

The I tried to allocate the free swap plus free RAM bytes of memory

char * p1 = new char[3072606208] ;
delete p1;

But this time code failed throwing the bad_alloc exception.Why the code didn't work this time.

Now I allocated the memory at compile time in a new program as shown below:

char p[3072606208] ;
char p2[4072606208] ;
char p3[5072606208];
cout<<"Size of array p = " <<sizeof p <<endl;
cout<<"Size of array p2 = " <<sizeof p2<<endl;
cout<<"Size of array p2 = " <<sizeof p3;

The out put shows

Size of array p = 3072606208
Size of array p1 = 4072606208
Size of array p2 = 777638912

Could you please help me understand what is happening here. Why did it allowed the memory to be allocated at the compile time but not at dynamically. When allocated compile time how come p and p1 were able to allocate memory greater than swap plus free RAM memory. Where as p2 failed. How exactly is this working. Is this some undefined behaviour or os specific behaviour. Thanks for your help. I am using Ubuntu 12.04 and gcc 4.6.3.

Answer 1

Memory pages aren't actually mapped to your program until you use them. All malloc does is reserve a range of the virtual address space. No physical RAM is mapped to those virtual pages until you try to read or write them.

Even when you allocate global or stack ("automatic") memory, there's no mapping of physical pages until you touch them.

Finally, sizeof() is evaluated at compile time, when the compiler has no idea what the OS will do later. So it will just tell you the expected size of the object.

You'll find that things will behave very differently if you try to memset the memory to 0 in each of your cases. Also, you might want to try calloc, which zeroes its memory.