JUnit 4 PermGen size overflow when running tests in Eclipse and Maven2

Question

I'm doing some unit tests with JUnit, PowerMock and Mockito. I have a lot of test classes annotated with @RunWith(PowerMockRunner.class) and @PrepareForTest(SomeClassesNames) to mock final classes and more than 200 test cases.

Recently I've run into a problem of a PermGen space overflow when I run my entire test suite in Eclipse or Maven2. When I run my test one by one then each of them succeeds.

I did some research about that, however none of the advice helped me (I have increased PermGenSize and MaxPermSize). Recently I've found out that there is one class that contains only static methods and each method returns object mocked with PowerMockito. I'm wondering whether it is a good practice and maybe this is the origin of the problem because static variables are being shared between unit tests?

Generally speaking is it a good practice to have a static class with a lot of static methods which returns static mocked objects?

Answer 1

I am getting PermGen errors from Junit in Eclipse too. But I am not using any mocking libs like Mockito nor EasyMock. However, my code base is large and my Junit tests are using Spring-Test (and are intense and complex test cases). For this, I need to truly increase the PermGen for all of my Junit tests.

Eclipse applies the Installed JRE settings to the Junit runs - not the eclipse.ini settings. So to change those:

• Window > Preferences > Java > Installed JRE's
• select the default JRE, Edit... button
• add to Default VM Arguments: -XX:MaxPermSize=196m

This setting will allow Junit tests to run the more intense TestCases in Eclipse, and avoid the OutOfMemoryError: PermGen. This should also be low risk because most simple Junit tests will not allocate all of that memory.

Answer 2

As @Brice says, the problems with PermGen will be coming from your extensive use of mocked objects. Powermock and Mockito both create a new class which sits between the class being mocked and your test code. This class is created at runtime and loaded into PermGen, and is (practically) never recovered. Hence your problems with the PermGen space.

To your question:

1) Sharing of static variables is considered a code smell. It's necessary in some cases, but it introduces depdendencies between tests. Test A needs to run before test B.

2) Usage of static methods to return a mocked object isn't really a code smell, it's a attern which is often used. If you really can't increase your permgen space, you have a number of options:

Use a pool of mocks, with PowerMock#reset() when the mock is put back into the pool. This would cut down on the number of creations you're doing.

Secondly, you said that your classes are final. If this is changeable, then you could just use an anonymous class in the test. This again cuts down on the amount of permgen space used:

Foo myMockObject = new Foo() {
     public int getBar() { throw new Exception(); }
}

Thirdly, you can introduce an interface (use Refactor->Extract Interface in Eclipse), which you then extend with an empty class which does nothing. Then, in your class, you do similar to the above. I use this technique quite a lot, because I find it easier to read:

public interface Foo {
  public int getBar();
}

public class MockFoo implements Foo {
  public int getBar() { return 0; }
}

then in the class:

Foo myMockObject = new MockFoo() {
     public int getBar() { throw new Exception(); }
}

I have to admit I'm not a particular fan of mocking, I use it only when necessary, I tend to either extend the class with an anonymous class or create a real MockXXX class. For more information on this point of view, see Mocking Mocking and Testing Outcomes. by Uncle Bob

By the way, in maven surefire, you can always forkMode=always which will fork the jvm for each test class. This won't solve your Eclipse problem though.

Answer 3

First : Mockito is using CGLIB to create mocks, and PowerMock is using Javassist for some other stuff, like removing the final markers, Powermock also loads classes in a new ClassLoader. CGLIB is known for eating the Permanent Generation (just google CGLIB PermGen to find relevant results on the matter).

It's not a straight answer as it depends on details of your project :

1. As you pointed there is static helper class, I don't know if holds static variables with mocks as well, I don't know the details of your code, so this is pure guess, and other readers that actually knows better might correct me.

   It could be the ClassLoader (and at least some of his childrens) that loaded this static class might be kept alive across tests - it might be because of statics (which lives in the Class realm) or because of some reference somewhere - that means that if the ClassLoader still lives (i.e. not garbage collected) his loaded classes are not discarded i.e. the classes including the generated ones are still in the PermGen.

2. These classes might also be huge in size, if you have a lot of these classes to be loaded this might be relevant to have higher PermGen values, especially since Powermock needs to reload classes in a new Classloader for each tests.

Again I don't know the details of your project, so I'm just guessing, but your permanent generation issue might be caused either due to point 1 or point 2, or even both.

Anyway generally speaking I would say yes : having a static class that might return static mocked object does look like a bad practice here, as it usually is in production code. If badly crafted it can leads to ClassLoader's leak (this is nasty!).

In practice I've seen running hundreds of tests (with Mockito only) without ever changing memory parameters and without seeing the CGLIB proxies being unloaded, and I'm not using static stuff appart the ones from the Mockito API.

If you are using a Sun/Oracle JVM you can try these options to track what's happening :

-XX:+TraceClassLoading and -XX:+TraceClassUnloading or -verbose:class

Hope that helps.

---

Outside the scope of this question :

Personnaly I don't like using to use Powermock anyway, I only use it in corner cases e.g. for testing unmodifiable legacy code. Powermock is too intrusive imho, it has to spawn for each test a new classloader to perform its deeds (modifying the bytecode), you have to heavily annotate the test classes to be able to mock, ... In my opinion for usual development all these little inconvenience outweight the benefit of the hability to mock finals. Even Johan the author of Powermock, once told me he was recommanding Mockito instead and keeping Powermock for some specific purpose.

Don't get me wrong here: Powermock is a fantastic piece of technology, that really help when you have to deal with (poorly) designed legacy code that you cannot change. But not for the every day developpement, especially if praticing TDD.