Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

java : How to use heap beyond 4 GB memory in 32 bit JVM

We have a product that runs currently on a 32 bit 1.6 JRE. We're using Berkeley DB which consumes about 2.5 GB RAM of the 4 GB address space. That leaves us about 750 MB of memory for the JVM address space.

We're currently hitting OutOfMemory issues with the current setup. It would be nice to increase our JVM heap size to 1.5 GB while still retaining Berkeley DB's 2.5 GB space. Is there any way to access more than 4 GB RAM/heap in a 32-bit JVM ? I've considering the following solutions
1) use a JVM which does better GC -- this will give me marginal results -- I could get about 50-100 MB working memory
2) something like memcached or "out of process ehcache " -- this can get me as much as the hardware allows with the overhead of IPC/serialization.

Are there other solutions to increasing an application's addressable memory ?

The solution should work on solaris 10 running sparc.

*UPDATE : Because of using native shared libraries, right now, we're unable to switch to a 64-bit JVM even though the OS is 64-bit *

thank you,

like image 754
anjanb Avatar asked Nov 10 '11 16:11

anjanb


People also ask

What is the max heap size for 32-bit JVM?

The maximum theoretical heap limit for the 32-bit JVM is 4G. Due to various additional constraints such as available swap, kernel address space usage, memory fragmentation, and VM overhead, in practice the limit can be much lower. On most modern 32-bit Windows systems the maximum heap size will range from 1.4G to 1.6G.

What is the maximum heap size of 32-bit and 64 bit JVM?

Max Heap Size. The maximum theoretical heap limit for the 32-bit and 64-bit JVM is easy to determine by looking at the available memory space, 2^32 (4 GB) for 32-bit JVM and 2^64 (16 Exabytes) for 64-bit JVM. In practice, due to various constraints, the limit can be much lower and varies given the operating system.

What is the maximum JVM heap size?

The theoretical limit is 2^64 bytes, which is 16 exabytes (1 exabyte = 1024 petabytes, 1 petabyte = 1024 terabytes). However, most OS's can't handle that. For instance, Linux can only support 64 terabytes of data. Note: We don't recommend you exceed 2 GB of in use JVM heap.


2 Answers

Are there other solutions to increasing an application's addressable memory ?

  1. Split single application in several processes with non-shared memory (not a threads; but processes). First process can run DB and second can run other part of the project. You can use RMI or shared memory or sockets to communicate between processes.
  2. Lower memory, reserved for OS. E.g. on x86-32 PAE allows OS to reserve a smaller than 1 GB of 4 GB virtual address space. (e.g "4GB / 4Gb split", supported on Oracle Linux)
  3. Put some data to the disk. The disk can be a RAM-disk to better speed; or it can be real disk and OS will speed up access to files using "page cache".

Also, every real SPARC (not ancient SuperSparc or poor-man LION) is 64-bit really. So, it can be easier to switch to 64-bit version of OS. I don't know about Solaris, but in linux there is possible to run 32-bit applications on top of 64-bit OS. And 64bit OS will allow you to run 64-bit JVM.

UPDATE: There are ramdisks in Solaris http://wikis.sun.com/display/BigAdmin/Talking+about+RAM+disks+in+the+Solaris+OS and I think you should try them for storing database (or temporary files of DB). There will be no extra serialization/IPC like in case (1); only extra read/write or mmap/munmap. But Ramdisk is order faster than SSD and 3-4 orders faster than HDD.

like image 66
osgx Avatar answered Nov 02 '22 05:11

osgx


32-bit programs are incapable of handling more than 4GB of memory addresses. They just don't have enough bits to represent more memory.

2^32 = 4 294 967 296

Your best bet would be to upgrade to a 64-bit JRE.

like image 40
StriplingWarrior Avatar answered Nov 02 '22 06:11

StriplingWarrior