Are mock frameworks and high test coverage important?

Question

Mock frameworks, e.g. EasyMock, make it easier to plugin dummy dependencies. Having said that, using them for ensuring how different methods on particular components are called (and in what order) seems bad to me. It exposes the behaviour to test class, which makes it harder to maintain production code. And I really don't see the benefit; mentally I feel like I've been chained to a heavy ball.

I much rather like to just test against interface, giving test data as input and asserting the result. Better yet, to use some testing tool that generates test data automatically for verifying given property. e.g. adding one element to a list, and removing it immediately yields the same list.

In our workplace, we use Hudson which gives testing coverage. Unfortunately it makes it easy to get blindly obsessed that everything is tested. I strongly feel that one shouldn't test everything if one wants to be productive also in maintenance mode. One good example would be controllers in web frameworks. As generally they should contain very little logic, testing with mock framework that controller calls such and such method in particular order is nonsensical in my honest opinion.

Dear SOers, what are your opinions on this?

Answer 1

I read 2 questions:

What is your opinion on testing that particular methods on components are called in a particular order?

I've fallen foul of this in the past. We use a lot more "stubbing" and a lot less "mocking" these days. We try to write unit tests which test only one thing. When we do this it's normally possible to write a very simple test which stubs out interactions with most other components. And we very rarely assert ordering. This helps to make the tests less brittle.

Tests which test only one thing are easier to understand and maintain.

Also, if you find yourself having to write lots of expectations for interactions with lots of components there could well be a problem in the code you're testing anyway. If it's difficult to maintain tests the code you're testing can often be refactored.

Should one be obsessed with test coverage?

When writing unit tests for a given class I'm pretty obsessed with test coverage. It makes it really easy to spot important bits of behaviour that I haven't tested. I can also make a judgement call about which bits I don't need to cover.

Overall unit test coverage stats? Not particularly interested so long as they're high.

100% unit test coverage for an entire system? Not interested at all.

Answer 2

I agree - I'm in favor of leaning heavily towards state verification rather than behavior verification (a loose interpretation of classical TDD while still using test doubles).

The book The Art of Unit Testing has plenty of good advice in these areas.

100% test coverage, GUI testing, testing getters/setters or other no-logic code, etc. seem unlikely to provide good ROI. TDD will provide high test coverage in any case. Test what might break.