Java: what determines the maximum max heap size possible in a linux machine

Question

I have two linux machines (both are VMs), one having 12GB memory and other having 8GB memory.

I tried to start the same java program on both machines, with maximum max heap size possible (using -Xmx flag). Following are the results I got.

• 12GB machine: 9460MB
• 8GB machine: 4790MB

If I specify a max heap size beyond above limits, I get below error.

Error occurred during initialization of VM
Could not allocate metaspace: 1073741824 bytes

I checked the free memory in two systems (using free command), and I got following.

• 12GB machine: approximately 3GB free.
• 8GB machine: approximately 4GB free.

My question is, what determines the maximum max heap size a java program can be started with, which would not result in above mentioned error? (System had sufficient memory to allocate 1073741824 bytes of memory when the program gave above error)

Answer 1

I have found interesting comments from JDK bug ( The bug in JDK 9 version and not in 8. It says bug was fixed in 8.x version but does not tell minor build number.

If virtual memory has been limited with "ulimit -v", and the server has a lot of RAM, then the JVM can not start without extra command line arguments to the GC.

// After "ulimit -v" The jvm does not start with default command line. 
$ ulimit -S -v 4194304
$ java -version
Error occurred during initialization of VM
Could not allocate metaspace: 1073741824 bytes

Comments:

The problem seems to be that we must specify MALLOC_ARENA_MAX.

If I set the environment variable MALLOC_ARENA_MAX=4, then the jvm can start without any extra arguments.

I guess that this is not something that can be fixed from the jvm. If so we can close this bug.

When using "UseConcMarkSweepGC" then the command line above does not work. I have tried to add MaxMetaspaceSize=128m, but it does not help. I am sure there are an argument that makes it work, but I have not found one. Configuring the GC with limited virtual memory is not very user friendly.

Change parameters to as per your requirement and try this one.

ulimit -S -v 4194304 
java -XX:MaxHeapSize=512m -XX:InitialHeapSize=512m -XX:CompressedClassSpaceSize=64m -XX:MaxMetaspaceSize=128m -XX:+UseConcMarkSweepGC -version

Answer 2

The memory you have available is the combination of free RAM plus swap space. It also depends on whether the system has overcommit enabled — if so, the kernel will allow programs to allocate more memory than is actually available (within reasonable limits), since programs often allocate more than they're really going to use.

Note that overcommit is enabled by default. To disable it, write 2 into /proc/sys/vm/overcommit_memory. (Strangely, a value of 0 does not mean "no overcommit".) But it's a good idea to read the overcommit documentation first.