Best way to allocate large memory

Question

In my Visual C++ app, I know the total objects(CMyObject) to be allocated is 16728064 and each object is 64 byte, so the total memory to be allocated is 1GB. The memory will be allocated in the beginning, used in the whole lifetime of the app, and release in the end.

In such a case, what is the best way to allocate the memory?

Current I try to allocate the memory at the beginning, as follows:

    CMyObject *p = new CMyObject[16728064];

    // Perform tasks.

    delete [] p;

But the allocation will fail for most of the time. Now I want to do as follows:

    CMyObject *p[10];

    p[0] = new CMyObject[1672806];

    p[1] = new CMyObject[1672806];

    …

    // Perform tasks

    Delete [] p[0];

    ….

This seems to work for some time.

Therefore, should I split the allocation into pieces as small as possible? Or are there any good solutions for such a situation?

Thanks

Answer 1

In general, yes you should split larger allocations into smaller fragments. Depending on your system, it may not have 1GB of contiguous memory.

Answer 2

Assuming this is X86 processor or something similar, only the virtual address space is contiguous. For X86, physical memory is composed of 4096 byte pages, and the physical pages do not have to be contiguous, only the mapped virtual address space.

When I run Windows XP 32 bit, on a system with 4GB, it shows 3.6 GB of physical memory available, and usually my test programs don't have a problem with allocating 1 GB, with failures to allocate memory occurring somewhere between 1.5GB and 2GB.

My guess is the reason for failure with large allocations of available physical memory has to do with the operating system as opposed to a processor virtual to physical mapping limitation.

What operating system are you using?