Extremely long garbage collection times

Question

We have a web application running Java 6, Tomcat 6, Spring Framework 3, Hibernate 4, EhCache. We have a problem with extremely long garbage collection times which can take 30 seconds or longer, leaving the application unresponsive.

We're currently in testing but apart from the obvious: add more memory, I was wondering if there are aspects we could tune to reduce garbage collection time.

The major contributor to memory use is EHCache as we are aggressively caching. But I always find it hard to size the EHCache stores (the new EhCache byte size stores, lead to all sorts of problems with us because the cached object graphs can be quite large).

These are my settings for the JVM

JAVA_OPTS="$JAVA_OPTS -server -Xms256m -Xmx704m XX:OnOutOfMemoryError=/usr/share/scripts/on_server_crash.sh -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/var/log/tomcat6 -XX:MaxPermSize=192m -XX:+UseConcMarkSweepGC

Answer 1

To reduce GC times, the best thing you can do is use off heap memory. If you can move as much of your large data as possible you can reduce your full GC time to as low as 10 milli-second even with 100s of MB of off heap memory. I believe Ehcache support off heap data stores, but if it doesn't or you can't use it I suggest you look at alternatives which do.

Given you only have a 700 MB maximum memory size it appears you are running on a server with very limited memory. Otherwise I would suggest you start with a maximum of 8 or 16 GB and reduce the memory size if you believe you don't really need it.

Answer 2

There is an excellent tool from FourSquare folks. Check this link and quick example they have. Foursquare Heap tool. . Based on diagnostics that you find in any of the above mentioned tools, most sorted solution to resolve the issue will be to either to add more RAM or add power to your CPU processor. If you are open to some infrastructure changes check Zing from Azul Systems. But I think the second option might be a stretch.