Is there a cookbook guide for GC problems?

Question

Almost everyone eventually runs into GC issues with Java.

Is there a cookbook guide or semi-automated tool to tune GC for Java?

My rationale is this:

• Almost anyone eventually has these problems
• There are many possible factors (say 20) out of which only a few affect your problem.
• Most people don't know how to identify the key factors so GC tuning is more like a black art than a science.
• Not everyone uses a HotSpot VM. Different Sun versions have different GC characteristics.
• There is little incentive to experiment (like run the VM with slightly different settings every day to see how they play out).

So the question really is: Is there something that I can use in a check-list manner? Or maybe even a tool that analyzes GC logs or heap dumps and gives me specific hints where to look (instead of telling me "95% of the data is allocated in objects of the type byte[]" which is basically useless).

Related questions:

• Appropriate Tomcat 5.5 start-up parameters to tune JVM for extremely high demand, large heap web application? which is very specific. My question is more wide.
• What are the best garbage collection settings for client side? Again very narrow scope
• Does anyone know of a good guide to configure GC in Java? HotSpot only
• JVM memory management & garbage collection book? is 80% there but I'm missing the checklist/cookbook/for-dummies approach.

Answer 1

Out of various resources I have compiled a sanity checklist that I use to analyze GC behavior and performance of my applications. These guidelines are general and apply to any vendor-specific JVM but contain also HotspotVM-specific information for illustration.

1. Disable Explicit GC. Explicit GC is a bad coding practice, it never helps. Use -XX:+DisableExplicitGC.

2. Enable Full GC logging. Lightweight yet powerful.

   • Compute Live Data Set, Allocation Rate, and Promotion Rate. This will tell you if you need a bigger Heap or if your eg. Young Gen is too small, or if your Survivor spaces are overflowing, etc.
   • Compute total GC time, it should be <5% of total running time.
   • Use -XX:+PrintTenuringDistribution -XX:+UnlockDiagnosticVMOptions -XX:+LogVMOutput -XX:LogFile=jvm.log -XX:+HeapDumpOnOutOfMemoryError -Xloggc:gc.log -XX:+PrintGCTimeStamps -XX:+PrintGCDetails -showversion

3. Consider additional means of collecting information about your GC. Logging is fine but there are sometimes available lightweight command-line tools that will give you even more insight. Eg. jstat for Hotspot which will show you occupation/capacity of Eden, Survivor and Old Gen.

4. Collect Class Histograms These are lightweigh and will show you the content of the heap. You can take snapshots whenever you notice some strange GC activity, or you can take them before/after Full GC:

   • Content of the OldGen space: You can find out which objects reside in the OldGen. You need to print histograms before and after Full GC. And since a YoungGen collection is executed before the Full GC, these Histograms will show you the content of the Old generation. Use -XX:+PrintClassHistogramBeforeFullGC -XX:+PrintClassHistogramAfterFullGC.
   • Detecting prematurely promoted objects: To determine if any instances are promoted early, you need to study the Histograms to see which classes are expected to reside in the OldGen and which classes should be seen only in the YoungGen. This cannot be done automatically, you need to reason about the purpose of each class and its instance to determine if the object is temporary or not.

5. Consider different GC Algorithm. The VMs usually come with several different GC implementations that are providing various tradeoffs : throughput, footprint, pause-less/short-pauses, real-time, etc. Consider the options you have and pick the one that suites your needs.

6. Beware of finalize(). Check that GC keeps up with classes using finalize(). The execution of this method may be quite costly and this can impact GC and application throughput.

7. Heap Dumps. This is the first step that is heavyweight and will impact the running application. Collect the Heap Dump to further study the heap content or to confirm a hypothesis observed in step 4.

Resources used:

Books:

• Java Performance - practical guide
• The Garbage Collection Handbook - theory explained

Talks/Articles:

• Java One 2012 Advanced JVM Tuning
• From Java code to Java heap
• Java One 2012 G1 Garbage Collector Performance Tuning
• Garbage Collection Tuning Guide

Mailing Lists:

• OpenJDK Hotspot GC Use

Answer 2

References for various GC information:

Oracle

Tuning Garbage Collection with the 5.0 Java[tm] Virtual Machine

and this also

Java SE 6 HotSpot[tm] Virtual Machine Garbage Collection Tuning

IBM

Fine Tuning Garbage Collection [link dead]

Extensible Verbose Toolkit

SAP JVM

Memory Management (Garbage Collection)

Detecting Memory Leaks

Detecting Hanging / Looping VMs

Analyzing Out-of-Memory Situations

Sorry I don't know much about SAP but have provided some things I have found.

As for a cookbook, tuning is most likely application specific at this level, but it is an interesting topic.

ADDENDUM

You also mentioned analysis tools. Some candidates are listed here:

Know of any Java garbage collection log analysis tools?