Understanding Groovy/Grails classloader leak

Question

Yesterday I deployed my first Grails (2.3.6) app to a dev server and began monitoring it. I just got an automated monitor stating that CPU was pinned on this machine, and so I SSHed into it. I ran top and discovered that it was my Java app's PID that was pinning the server. I also noticed memory was at 40%. After a few seconds, the CPU stopped pinning, went down to a normal level, and memory went back down into the ~20% range. Classic major GC.

While it was collecting, I did a heap dump. After the GC, I then opened the dump in JVisualVM and saw that most of the memory was being allocated for an org.codehaus.groovy.runtime.metaclass.MetaMethodIndex.Entry class. There were almost 250,000 instances of these in total, eating up about 25 MB of memory.

I googled this class and took a look at it's ultra helpful Javadocs. So I still have no idea what this class does.

But googling it also brought up about a dozen or so related articles (some of them SO questions) involving this class and a PermGen/classloader leak with Grails/Groovy apps. And while it seems that my app did in fact clean up these 250K instance with a GC, it still is troubling that there were so many instances of it, and that the GC pinned CPU for over 5 minutes.

My questions:

• What is this class and what is Groovy doing with it?
• Can someone explain this answer to me? Why would -XX:+CMSClassUnloadingEnabled -XX:+CMSPermGenSweepingEnabled help this particular problem?
• Why is this class particularly troublesome for the PermGen?

Answer 1

Groovy is a dynamic language, every method call is dispatched dynamically. To optimise that Groovy creates a MetaClass for every java.lang.Class in the MetaClassRegistry. These MetaClass instances are created on-demand and stored using Weak references.

The reason you see a lot of org.codehaus.groovy.runtime.metaclass.MetaMethodIndex.Entry is because Groovy is storing a map of classes and methods in memory so that they can be quickly dispatched by the runtime. Depending on the size of the application this can be as you have discovered thousands of classes as each class can have dozens sometimes hundreds of methods.

However, there is no "memory leak" in Groovy and Grails, what you are seeing is normal behaviour. Your application is running low on memory, probably because it hasn't been allocated enough memory, this in turn causes MetaClass instances to be garbage collected. Now say for example you have a loop:

for(str in strings) {
   println str.toUpperCase()
}

In this case we are calling a method on the String class. If you are running low on memory what will happen is that for each iteration of the loop the MetaClass will be garbage collected and then recreated again for the next iteration. This can dramatically slow down an application and lead to the CPU being pinned as you have seen. This state is commonly referred to as "metaclass churn" and is a sign your application is running low on heap memory.

If Groovy was not garbage collecting these MetaClass instances then yes that would mean there is a memory leak in Groovy, but the fact that it is garbage collecting these classes is a sign that all is well, except for the fact that you have not allocated enough heap memory in the first place. That is not to say that there may be a memory leak in another part of the application that is eating up all the available memory and leaving not enough for Groovy to operate correctly.

As for the other answer you refer to, adding class unloading and PermGen tweaks won't actually do anything to resolve your memory issues unless you dynamically parsing classes at runtime. PermGen space is used by the JVM to store dynamically created classes. Groovy allows you to compile classes at runtime using GroovyClassLoader.parseClass or GroovyShell.evaluate. If you are continuously parsing classes then yes adding class unloading flags can help. See also this post:

Locating code that is filling PermGen with dead Groovy code

However, a typical Grails application does not dynamically compile classes at runtime and hence tweaking PermGen and class unloading settings won't actually achieve anything.

You should verify if you have allocated enough heap memory using the -Xmx flag and if not allocate more.