Malloc Memory Questions

Question

First of all I noticed when I malloc memory vs. calloc the memory footprint is different. I am working with datasets of several GB. It is ok for this data to be random.

I expected that I could just malloc a large amount of memory and read whatever random data was in it cast to a float. However, looking at the memory footprint in the process viewer the memory is obviously not being claimed (vs. calloc where I see a large foot print). I ran a loop to write data into the memory and then I saw the memory footprint climb. Am I correct in saying that the memory isn't actually claimed until I initialize it?

Finally after I passed 1024*1024*128 bytes (1024 MB in the process viewer) I started getting segfaults. Calloc however seems to initialize the full amount up to 1 GB. Why do I get segfaults when initializing memory in a for loop with malloc at this number 128MB and why does the memory footprint show 1024MB?

If malloc a large amount from memory and then read from it what am I getting (since the process viewer shows almost no footprint until I initialize it)?

Finally is there any way for me to alloc more than 4GB? I am testing memory hierarchy performance.

Code for #2:

    long long int i;
    long long int *test=(long long int*)malloc(1024*1024*1024);
    for (i=0;i<1024*1024*128;i++)
            test[i]=i;

    sleep(15);

Answer 1

Some notes:

1. As the comments note, Linux doesn't actually allocate your memory until you use it.
2. When you use calloc instead of malloc, it zeroes out all the memory you requested. This is equivalent to using it.