64-bit JVM limited to 300GB of memory?

Question

I am attempting to run a Java application on a cluster computing environment (IBM LSF running CentOS release 6.2 Final) that can provide me with up to 1TB of RAM space.

I could create a JVM with up to 300GB of maximum memory (Xmx), although I need more than that (I can provide details, if requested).

However, it seems to be impossible to create a JVM with more than 300GB of maximum memory using the Xmx option. To be more specific, I get the classic error message:

Error occurred during initialization of VM.

Could not reserve enough space for object heap.

The details of my (64-bit) JVM are below:

OpenJDK Runtime Environment (IcedTea6 1.10.6) (rhel-1.43.1.10.6.el6_2-x86_64)

OpenJDK 64-Bit Server VM (build 20.0-b11, mixed mode)

I've also tried with a Java 7 64-bit JVM but I've had exactly the same problem.

Moreover, I tried to create a JVM to run a HelloWorld.jar, but still JVM creation fails if you ask for more than -Xmx300G, so I don't think it has anything to do with the specific application.

---

Does anyone have any idea why I cannot create a JVM with more than 300G of max memory?

Can anyone please suggest a solution/workaround?

Answer 1

I can think of a couple of possible explanations:

• Other applications on your system are using so much memory that there isn't 300Gb available right now.

• There could be a resource limit on the per-process memory size. You can check this using ulimit. (Note that according to this bug, you will get the error message if the per-process resource limit stops the JVM allocating the heap regions.)

• It is also possible that this is an "over commit" issue; e.g. if your application is running in a virtual and the system as a whole cannot meet the demand because there is too much competition from other virtuals.

---

A couple of the other ideas suggested are (IMO) unlikely:

• Switching the JRE is unlikely to make any difference. I've never heard or seen of arbitrary memory limits in specific 64 bit JVMs.

• It is unlikely to be due to not having enough contiguous memory. Certainly contiguous physical memory is not required. The only possibility might be contiguous space on the swap device, but I don't recall that being an issue for typical Linux OSes.

---

Can anyone please suggest a solution/workaround?

• Check the ulimit.

• Write a tiny C program that attempts to malloc lots of memory and see how much that can allocate before it fails.

• Ask the system (or hypervisor) administrator for help.