How do I write "dirty" unit tests?

Question

I am reading Code Complete. In that book, Steve McConnell warns that "Developer tests tend to be 'clean tests.' Developers tend to test for whether the code works (clean test) rather than test for all the ways the code breaks (dirty tests)."

How do I write a test for the way the code breaks? I mean, I can write tests for bad input and make sure that it is blocked correctly. But aside from that, what sorts of things should I be thinking about? What does McConnell mean here? I am comfortable with basic unit testing but trying to master it.

Answer 1

I think you are on the right path here. Tests to prove that the code works would call a method with sensible, meaningful and expected inputs, with the program in normal state. While tests to break the code try to think "out of the box" regarding that piece of code, thus use any sort of senseless or unexpected input.

What is IMHO important though is to understand that the two thought processes are very different. When a developer writes code in TDD fashion, (s)he tends to focus on the various bits of functionality to implement in the code, and the tests to prove that this and that bit of functionality or use case works as specified. Tests created this way are what McConnell calls "clean tests".

Thinking about how a piece of code could be broken requires a very different thought process and different experience too. It requires looking at your methods and APIs from a different angle, e.g. temporarily putting aside what you know about the aim of these methods and parameters, and focusing only what is technically possible to do with them. Also to think about all the - often implied - preconditions or dependencies required for this method to work correctly. Does it depend on a configuration parameter read from DB? Does it write to the file system? Does it call another component, expecting it to having been initialized properly beforehand? Does it use large amounts of memory? Does it display a message on a GUI?... And what if one or more of these doesn't hold?

All this leads to important questions: how should your method handle such dirty cases? Should it crash? Throw an exception? Continue as best as it can? Return an error code? Log an error report?... All these little or bigger decisions are actually quite important for defining the contract of a method or an API correctly and consistently.

Kent Beck talks about switching between wearing "developer hat" and "tester hat" in the same sense. Fluently switching viewpoints and thought processes this way requires practice and experience.