java does not properly reserve huge initial heap size

Question

I'm working on a single machine with 512GB RAM (addressed by several AMD Opteron 6212 CPUs). There is currently about 300GB RAM free. Running a large java computation by running

java path/to/myApp -Xms280g -Xmx280g > output.txt

should make Java reserve 280GB immediately, and error if that fails. Strangely, no error occurs but top only shows a memory usage of 30.4GB but it doesn't crash. How can this happen? Isn't java supposed to crash if the initial heap size cannot be allocated?

And effectively, I get OutOfMemory/Java heap space/GC overhead limit errors once the 30.4GB are full, well before the 280GB is ever reached. Running with 250GB or 300GB yields a similar 30.3GB ~ 30.4GB limit. I'm running OpenJDK 64-bit server VM with OpenJDK Runtime Environment (IcedTea6) on Gentoo Linux, and there is plenty of free RAM (over 300GB).

Answer 1

The order of the parameters is incorrect. You are passing -Xms280g -Xmx280g as arguments to your own program and not to the JVM. The correct is:

java -Xms280g -Xmx280g path/to/myApp

Answer 2

If you want the memory specified in -Xms to be grabbed during the initialization of your application then use

java -XX:+AlwaysPreTouch -Xms2G -Xmx2G ......

AlwaysPreTouch will demand every memory page during the initialization of the JVM rather than just keeping what is not needed as “virtual”. However, note that this will have some latency in the starting of the JVM.

Use the aforementioned switch and then check with top. You will get full 2G (in fact a little more) for your JVM.