Automated memory leak detection in Java

Question

I was thinking about automated memory leak detection for a Java program. The basic algorithm is to create JUnits that contain the following logic:

Call System.gc() several times
Determine initial heap memory consumption using either Runtime class or JMX
Loop 
    Do something that exercises program under test
End loop

Call System.gc() several times
Determine final heap memory consumption
Compare initial and final memory numbers

The loop is being used to see if memory is creeping up in small increments.

It is necessary to distinguish between expected and unexpected increases in memory use.

This is not really a unit test. But the JUnit framework is convenient to use.

Do you think that this approach is valid? Do you think that this approach will be sucessful in identifying memory leaks? Have you ever done something like this?

Answer 1

I developed a simple unit test framework for memory leaks which has worked reliably for me. The basic idea is to create a weak reference to an object which should be garbage collected, execute the test, perform a full GC, and then verify that the weak reference has been cleared.

Here is a fairly typical regression test using my framework:

public void testDS00032554() throws Exception {
  Project testProject = getTestProject();
  MemoryLeakVerifier verifier = new MemoryLeakVerifier(new RuntimeTestAction(getTestClassMap()));
  testProject.close();
  verifier.assertGarbageCollected("RuntimeTestAction should be garbage collected when project closed");
}

There are some things to note here:

1. It is critical that the object you wish to have be collected should not be stored in a variable in your unit test as it will be retained through the end of your test.
2. This is a useful technique for regression tests where a leak has been reported and you know which object should have been removed.
3. One problem with this approach is that it is hard to determine why the test failed. At this point you will need a memory profiler (I am partial to YourKit). However IMO it is still useful to have the regression tests so that the leaks cannot be accidentally reintroduced in the future.
4. I ran into some threading problems with not all references being cleared immediately, so the method now tries performing the GC a number of times before failing (as described in this article: Java Tip 130: Do you know your data size?)

Here's the full helper class in case you want to try it out:

/**
 * A simple utility class that can verify that an object has been successfully garbage collected.
 */
public class MemoryLeakVerifier {
private static final int MAX_GC_ITERATIONS = 50;
private static final int GC_SLEEP_TIME     = 100;

private final WeakReference reference;

public MemoryLeakVerifier(Object object) {
    this.reference = new WeakReference(object);
}

public Object getObject() {
    return reference.get();
}

/**
 * Attempts to perform a full garbage collection so that all weak references will be removed. Usually only
 * a single GC is required, but there have been situations where some unused memory is not cleared up on the
 * first pass. This method performs a full garbage collection and then validates that the weak reference
 * now has been cleared. If it hasn't then the thread will sleep for 50 milliseconds and then retry up to
 * 10 more times. If after this the object still has not been collected then the assertion will fail.
 *
 * Based upon the method described in: http://www.javaworld.com/javaworld/javatips/jw-javatip130.html
 */
public void assertGarbageCollected(String name) {
    Runtime runtime = Runtime.getRuntime();
    for (int i = 0; i < MAX_GC_ITERATIONS; i++) {
        runtime.runFinalization();
        runtime.gc();
        if (getObject() == null)
            break;

        // Pause for a while and then go back around the loop to try again...
        try {
            EventQueue.invokeAndWait(Procedure.NoOp); // Wait for the AWT event queue to have completed processing
            Thread.sleep(GC_SLEEP_TIME);
        } catch (InterruptedException e) {
            // Ignore any interrupts and just try again...
        } catch (InvocationTargetException e) {
            // Ignore any interrupts and just try again...
        }
    }
    PanteroTestCase.assertNull(name + ": object should not exist after " + MAX_GC_ITERATIONS + " collections", getObject());
}

}