Maximum amount of memory per Java process on Windows?

Question

What is the maximum heap size that you can allocate on 32-bit Windows for a Java process using -Xmx?

I'm asking because I want to use the ETOPO1 data in OpenMap and the raw binary float file is about 910 MB.

Answer 1

There's nothing better than an empirical experiment to answer your question. I've wrote a Java program and run it while specifying the XMX flag (also used XMS=XMX to force the JVM pre-allocate all of the memory). To further protect against JVM optimizations, I've actively allocate X number of 10MB objects. I run a number of test on a number of JVMs increasing the XMX value together with increasing the number of MB allocated, on a different 32bit operating systems using both Sun and IBM JVMs, here's a summary of the results:

OS:Windows XP SP2, JVM: Sun 1.6.0_02, Max heap size: 1470 MB
OS: Windows XP SP2, JVM: IBM 1.5, Max heap size: 1810 MB
OS: Windows Server 2003 SE, JVM: IBM 1.5, Max heap size: 1850 MB
OS: Linux 2.6, JVM: IBM 1.5, Max heap size: 2750 MB

Here's the detailed run attempts together with the allocation class helper source code:

WinXP SP2, SUN JVM:

 C:>java -version java version "1.6.0_02" Java(TM) SE Runtime Environment (build 1.6.0_02-b06) Java HotSpot(TM) Client VM (build 1.6.0_02-b06, mixed mode)  
java -Xms1470m -Xmx1470m Class1 142 ... about to create object 141 object 141 created
  
C:>java -Xms1480m -Xmx1480m Class1 145 Error occurred during initialization of VM Could not reserve enough space for object heap Could not create the Java virtual machine. 

WinXP SP2, IBM JVM

  C:>c:\ibm\jdk\bin\java.exe -version java version "1.5.0" Java(TM) 2 Runtime Environment, Standard Edition (build pwi32devifx-20070323 (if ix 117674: SR4 + 116644 + 114941 + 116110 + 114881)) IBM J9 VM (build 2.3, J2RE 1.5.0 IBM J9 2.3 Windows XP x86-32 j9vmwi3223ifx-2007 0323 (JIT enabled) J9VM - 20070322_12058_lHdSMR JIT  - 20070109_1805ifx3_r8 GC   - WASIFIX_2007) JCL  - 20070131  
c:\ibm\jdk\bin\java.exe -Xms1810m -Xmx1810m Class1 178 ... about to create object 177 object 177 created
  
C:>c:\ibm\jdk\bin\java.exe -Xms1820m -Xmx1820m Class1 179 JVMJ9VM015W Initialization error for library j9gc23(2): Failed to instantiate he ap.  1820M requested Could not create the Java virtual machine. 

Win2003 SE, IBM JVM

 C:>"C:\IBM\java" -Xms1850m -Xmx1850m Class1 sleeping for 5 seconds. Done.  
C:>"C:\IBM\java" -Xms1880m -Xmx1880m  Class1 JVMJ9VM015W Initialization error for library j9gc23(2): Failed to instantiate he ap.  1880M requested Could not create the Java virtual machine.

Linux 2.6, IBM JVM

 [root@myMachine ~]# /opt/ibm/java2-i386-50/bin/java -version java version "1.5.0" Java(TM) 2 Runtime Environment, Standard Edition (build pxi32dev-20060511 (SR2)) IBM J9 VM (build 2.3, J2RE 1.5.0 IBM J9 2.3 Linux x86-32 j9vmxi3223-20060504 (JIT enabled) J9VM - 20060501_06428_lHdSMR JIT  - 20060428_1800_r8 GC   - 20060501_AA) JCL  - 20060511a  
/opt/ibm/java2-i386-50/bin/java -Xms2750m -Xmx2750m Class1 270
  
[root@myMachine ~]# /opt/ibm/java2-i386-50/bin/java -Xms2800m -Xmx2800m Class1 270 JVMJ9VM015W Initialization error for library j9gc23(2): Failed to instantiate heap.  2800M requested Could not create the Java virtual machine. 

Here's the code:

 import java.util.StringTokenizer;   public class Class1 {          public Class1() {}          private class BigObject {                 byte _myArr[];                 public BigObject() {                         _myArr = new byte[10000000];                 }         }     public static void main(String[] args) {                 (new Class1()).perform(Integer.parseInt(args[0]));         }         public void perform(int numOfObjects) {                 System.out.println("creating 10 MB arrays.");                 BigObject arr[]  = new BigObject[numOfObjects];                 for (int i=0;i <numOfObjects; i++) {                         System.out.println("about to create object "+i);                         arr[i] = new BigObject();                         System.out.println("object "+i+" created");                 }                 System.out.println("sleeping for 5 seconds.");                 try {                 Thread.sleep(5000);                 }catch (Exception e) {e.printStackTrace();}                 System.out.println("Done.");     }  } 

Answer 2

For a large file I suggest you use a memory mapped file. This doesn't use heap space (or very little) so maximum heap size shouldn't be a problem in this case.