Java memory usage on Linux

Question

I'm running a handfull of Java Application servers that are all running the latest versions of Tomcat 6 and Sun's Java 6 on top of CentOS 5.5 Linux. Each server runs multiple instances of Tomcat.

I'm setting the -Xmx450m -XX:MaxPermSize=192m parameters to control how large the heap and permgen will grow. These settings apply to all the Tomcat instances across all of the Java Application servers, totaling about 70 Tomcat instances.

Here is a typical memory usage of one of those Tomcat instances as reported by Psi-probe

Eden           = 13M
Survivor       = 1.5M 
Perm Gen       = 122M 
Code Cache     = 19M 
Old Gen        = 390M 
Total          = 537M

CentOS however is reporting RAM usage for this particular process at 707M (according to RSS) which leaves 170M of RAM unaccounted for.

I am aware that the JVM itself and some of it's dependancy libraries must be loaded into memory so I decided to fire up pmap -d to find out their memory footprint. According to my calculations that accounts for about 17M.

Next there is the Java thread stack, which is 320k per thread on the 32 bit JVM for Linux. Again, I use Psi-probe to count the number of threads on that particular JVM and the total is 129 threads. So 129 + 320k = 42M

I've read that NIO uses memory outside of the heap, but we don't use NIO in our applications.

So here I've calculated everything that comes to (my) mind. And I've only accounted for 60M of the "missing" 170M.

What am I missing?

Answer 1

Try using the incremental garbage collector, using the -Xincgc command line option. It's little more aggressive on the whole GC efforts, and has a special happy little anomaly: it actually hands back some of its unused memory to the OS, unlike the default and other GC choices ! This makes the JVM consume a lot less memory, which is especially good if you're running multiple JVM's on one machine. At the expense of some performance - but you might not notice it. The incgc is a little secret it seems, because noone ever brings it up... It's been there for eons (90's even).