Optimizing Tomcat / Garbage Collection

Question

Our server has 128GB of RAM and 64 cores, running Tomcat 7.0.30 and Oracle jdk1.6.0_38, on CentOS 6.3.

Every 60 minutes we're seeing garbage collection that was taking 45 - 60 seconds. Adding -XX:-UseConcMarkSweepGC increased page load times by about 10% but got that down to about 3 seconds, which is an acceptable trade-off.

Our config:

-Xms30g
-Xmx30g
-XX:PermSize=8g
-XX:MaxPermSize=8g
-Xss256k
-XX:-UseConcMarkSweepGC

We set the heap at 30 GB to keep 32 bit addressing (I read that above 32 GB the 64 bit addressing takes up more memory, so you have to go to about 48 GB to see improvements).

Using VisualGC I can see that the Eden space is cycling through every 30 - 60 minutes, but not much happens with the Survivor 0, Survivor 1, Old Gen, and Perm Gen.

We have a powerful server. What other optimizations can we make to further decrease the 3 second GC time?

Any recommendations to improve performance or scaling?

Any other output or config info that would help?

Answer 1

It might sound counter-intuitive, but have you tried allocating a lot less memory? E.g. do you really need a 30G heap? In case you can get along with 4G or even less: Garbage collection might be more frequent, but when it happens it will be a lot faster. Typically I find this more desirable than allocating a lot of memory, suffering from the time it takes to clean it up.

Even if this will not help you because you really need 30G of memory, others might come along with a similar problem and they might benefit from allocating less.

Answer 2

Seems that you need Incremental GC to reduce pauses:

• -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode

and for tracing without visualgc this always went well for me (output in catalina.out):

• -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps

2013-01-05T22:52:13.954+0100: 15918369.557: [GC 15918369.557: [DefNew: 65793K->227K(98304K), 0.0031220 secs] 235615K->170050K(491520K), 0.0033220 secs] [Times: user=0.01 sys=0.00, real=0.00 secs]

After you can play with this:

• -XX:NewSize=ABC -XX:MaxNewSize=ABC
• -XX:SurvivorRatio=ABC
• -XX:NewRatio=ABC

Reference: Virtual Machine Garbage Collection Tuning