Allocating memory about 10GB on heap [duplicate]

Question

I was checking how big of an array can I create on a X64 application, my understanding was that I can create arrays bigger than 2^31 on X64 process but I'm getting a compilation error on VS2010 compiler, below code

const size_t ARRSIZE = size_t(1)<<32;
int main()
{
    char *cp = new char[ARRSIZE];
    return 0;
}

gives compiler error "error C2148: total size of array must not exceed 0x7fffffff bytes" on VS2010 on target x64 platform, I can create upto (size_t(1)<<32 - 1);

I have Linker->Advanced->Target Machine is Machinex64. Also Linker->System->Enable Large Addresses as Yes ( Not sure if this really matters ). Does the paging file\Physical Ram present in the pc matter here? (I'm sure that it is a 64-bit app because if I remove that line and just have char* cp; it is 8-byte.) Am I missing some settings?

Answer 1

This appears to be a defect in the 32-bit cross compiler for x64 targets. The Microsoft Connect link posted by icabod in the comments above addresses this particular issue. Unfortunately the bug's status has been set to Closed - Won't Fix.

The following code snippets will fail to compile using the 32-bit cross compiler for x64:

char* p = new char[(size_t)1 << 32];

and

const size_t sz = (size_t)1 << 32;
char* p = new char[sz];

Both of the above will fail with the error message error C2148: total size of array must not exceed 0x7fffffff bytes when compiled with the 32-bit cross compiler for x64. Unfortunately, Visual Studio does launch the 32-bit compiler even when run on a 64-bit version of Windows, targetting x64.

The following workarounds can be applied:

• Compiling the code using the native 64-bit compiler for x64 from the command line.

• Changing the code to either of the following:

  size_t sz = (size_t)1 << 32;  // sz is non-const
  char* p = new char[sz];
  

  or

  std::vector<char> v( (size_t)1 << 32 );
  

The bug is still present in Visual Studio 2012, and all workarounds still apply.

Answer 2

The compiler is likely trying to optimize since your ARRSIZE value is a constant. And then it hits its own static initialization limit. You could probably just take out the "const" keyword and it will work.

If not, something like this will likely work.

extern size_t GetArraySize();

int main()
{
    size_t allocationsize = GetArraySize();

    char *cp = new char[allocationsize];
    return 0;
}

size_t GetArraySize()
{
    // compile time assert to validate that size_t can hold a 64-bit value
    char compile_time_assert_64_bit[(sizeof(size_t) == 8)?1:-1];

    size_t allocsize = 0x100000000UL; // 64-bit literal

    return allocsize;
}