Java throwing out of memory exception before it's really out of memory?

Question

I wish to make a large int array that very nearly fills all of the memory available to the JVM. Take this code, for instance:

    final int numBuffers = (int) ((runtime.freeMemory() - 200000L) / (BUFFER_SIZE));
    System.out.println(runtime.freeMemory());
    System.out.println(numBuffers*(BUFFER_SIZE/4)*4);
    buffers = new int[numBuffers*(BUFFER_SIZE / 4)];

When run with a heap size of 10M, this throws an OutOfMemoryException, despite the output from the printlns being:

9487176
9273344

I realise the array is going to have some overheads, but not 200k, surely? Why does java fail to allocate memory for something it claims to have enough space for? I have to set that constant that is subtracted to something around 4M before Java will run this (By which time the printlns are looking more like: 9487176 5472256 )

Even more bewilderingly, if I replace buffers with a 2D array:

buffers = new int[numBuffers][BUFFER_SIZE / 4];

Then it runs without complaint using the 200k subtraction shown above - even though the amount of integers being stored is the same in both arrays (And wouldn't the overheads on a 2D array be larger than that of a 1D array, since it's got all those references to other arrays to store).

Any ideas?

Answer 1

The VM will divide the heap memory into different areas (mainly for the garbage collector), so you will run out of memory when you attempt to allocate a single object of nearly the entire heap size.

Also, some memory will already have been used up by the JRE. 200k is nothing with todays memory sizes, and 10M heap is almost unrealistically small for most applications.

The actual overhead of an array is relatively small, on a 32bit VM its 12 bytes IIRC (plus what gets wasted if the size is less than the minimal granularity, which is AFAIK 8 bytes). So in the worst case you have something like 19 bytes overhead per array.

Note that Java has no 2D (multi-dimensional) arrays, it implements this internally as an array of arrays.