Do setup/teardown hurt test maintainability?

Question

This seemed to spark a bit of conversation on another question and I thought it worthy to spin into its own question.

The DRY principle seems to be our weapon-of-choice for fighting maintenance problems, but what about the maintenance of test code? Do the same rules of thumb apply?

A few strong voices in the developer testing community are of the opinion that setup and teardown are harmful and should be avoided... to name a few:

• James Newkirk
• Jay Fields, [2]

In fact, xUnit.net has removed them from the framework altogether for this very reason (though there are ways to get around this self-imposed limitation).

What has been your experience? Do setup/teardown hurt or help test maintainability?

UPDATE: do more fine-grained constructs like those available in JUnit4 or TestNG (@BeforeClass, @BeforeGroups, etc.) make a difference?

Answer 1

The majority (if not all) of valid uses for setup and teardown methods can be written as factory methods which allows for DRY without getting into issues that seem to be plagued with the setup/teardown paradigm.

If you're implementing the teardown, typically this means you're not doing a unit test, but rather an integration test. A lot of people use this as a reason to not have a teardown, but IMO there should be both integration and unit test. I would personally separate them into separate assemblies, but I think a good testing framework should be able to support both types of testing. Not all good testing is going to be unit testing.

However, with the setup there seems to be a number of reasons why you need to do things before a test is actually run. For example, construction of object state to prep for the test (for instance setting up a Dependency Injection framework). This is a valid reason for a setup, but could just as easily be done with a factory.

Also, there is a distinction between class and method level setup/teardown. That needs to be kept in mind when considering what you're trying to do.

My biggest problem that I have had with using the setup/teardown paradigm is that my tests don't always follow the same pattern. This has brought me into using factory patterns instead, which allows me to have DRY while at the same time being readable and not at all confusing to other developers. Going the factory route, I've been able to have my cake and eat it to.