How to ensure quality of junit tests?

Question

Are there proven ways of verifying quality of junit tests or integration tests?

Should your business analyst review unit tests to cerfity? Or are there any other ways? In the traditional code first environment a peer or lead would review the test plan but how about automated tests?

I looked at this stackflow thread but couldn't extract anything meaningful stuff.

Thoughts?

Answer 1

Mutation testing and code coverage can verify the quality of your tests.

So first check than your code coverage is high enough. After this verify with mutation testing than your test are good. Mutation testing tool makes small change(s) in production code and reruns test - after a modification a good test should fail. For mutation testing tool in Java look at PIT Mutation Testing and this blog post: Introduction to mutation testing with PIT and TestNG

But this is still not enough, tests should be good written and readable. So you need code review also and quality rules verification for tests. I recommend nice book about writing good tests Practical Unit Testing. Chapter 10: Maintainable tests from this book is available for free.

Answer 2

Here's a nice linked article:

http://www.ibm.com/developerworks/java/library/j-cq01316/index.html?ca=drs

And:

Good Tests ⇒ High Coverage High Coverage ⇒/⇒ Good Tests

Coverage tools are useful to identify what areas of your project need more attention, but it doesn't mean that areas with good coverage shouldn't need more attention.

Answer 3

Code coverage tool is a good start, but knowing that a given line was executed does not mean it was tested. Infamous test cases without assertions or expected=Exception.class are an example.

I can imagine few criteria on this level:

• if the line is tested, any change to it (inverting condition, removing...) should fail at least one test
• given piece of logic should be fully reconstructible based only its tests
• the test does not mirror the production code
• the test should not be dependent on current date, locale, timezone, order of other tests

One might try to automate the first one, others are more or less subjective.

As for analyst doing test review - probably only Fitensse fixtures are readable enough to satisfy non-developers.

Answer 4

Code review is the best way to ensure test quality. I would not have business analysts review the tests, for the simple fact that they might not have the training necessary to understand the tests. Also, unit tests do not all live at the functional level, where analysts' requirements are. An analyst might say 'when the user clicks save, the profile is saved' whereas you might have to write n number of tests across multiple layers to get that functionality.