Java unit testing - proper isolation?

Question

I'd like to know if my approach to unit testing has been wrong:

My application has a boot strap process that initializes a several components and provides services to various sub-systems - let's call it the "controller".

In many cases, in order to unit test these sub-systems, I would need access to the controller as these sub-systems may depend on it. My approach to doing this unit test would be to initialize the system, and then provide the controller to any unit test that requires it. I achieve this via inheritance: I have a base unit test that initializes and tests the controller, then any unit test requiring the controller would extend this base class, and hence, have access to it.

My question is this:

(1) Is this achieving proper isolation? It makes sense to me that unit tests should be done in isolation so that they are repeatable and independent - is it ok that I am providing a real initialized controller rather than mocking it or attempting to mock the specific environment required by each test?

(2) As a best practice (assuming that my previous approach is OK) - should I be creating the controller over and over for each unit test, or would it suffice to create it once (its state is not changing).

Answer 1

If we are supplying a "real" controller to test another component, then strictly speaking we are performing an integration test rather than a unit test. This is not necessarily a bad thing, but consider the following points:

Cost of Creating the Controller

If the controller is a heavyweight object with a considerable cost to construct, then every unit test will incur this cost. As the number of unit tests grows in number, that cost may begin to dominate the overall test execution time. It is always desirable to keep the runtime of unit tests as small as possible to allow quick turnaround after code changes.

Controller Dependencies

If the controller is a complex object, it may have dependencies of its own that need to be instantiated in order to construct the controller itself. For example, it may need to access a database or configuration file of some kind. Now, not only does the controller need to be initialized, but also those components. As the application evolves over time, the controller may require more and more dependencies, just making this problem worse as time goes on.

Controller State

If the controller carries any state, the execution of a unit test may change that state. This, in turn, may change the behaviour of subsequent unit tests. Such changes may result in apparently non-deterministic behaviour of the unit tests, introducing the possibility of masking bugs. The cure for this problem is to create the controller anew for each test, which may be impractical if that creation is expensive (as noted above).

Combinatorial Problem

The number of combinations of possible inputs to the composite system of the unit under test and the controller object may be much larger than the number of combinations for the unit alone. That number might be too large to test practically. By testing the unit in isolation with a stub or mock object in place of the controller, it is easier to keep the number of combinations under control.

God Object

If the controller is conveniently accessible to all components in every unit test, there will be a great temptation to turn the controller into a God Object that knows everything about every component in the system. Even worse, those components may begin to interact with one another through that god object. The end result is that the separation between application components begins to erode and system starts to become monolithic.

Technical Debt

Even if the controller is stateless and cheap to instantiate today, that may change as the application evolves. If that day arrives after we have written a large number of unit tests, we might be faced with a large refactoring exercise of all of those tests. Furthermore, the actual system code might also need refactoring to replace all of the controller references with lighter weight interfaces. There is a risk that the refactoring cost is significant -- possibly even too high to contemplate, resulting in a system is "stuck" in an undesirable form.

Recommendation

In order to avoid these pitfalls now and in the future, my recommendation is to avoid supplying the real controller to the unit tests.

The full controller is likely to be difficult to stub or mock effectively. This will induce (desirable) pressure to express a component's dependencies as a "thin", focused interface in place of the "thick", "kitchen sink" interface that the controller is likely to present. Why is this desirable? It is desirable because this practice promotes better separation of concerns between system components, yielding architectural benefits far beyond the unit test code base.

For lots of good practical advice about how to achieve separation of concerns and generally write testable code, see Misko Hevery's guide and talks.