When is a Test not a Unit-test?

Question

See Michael Feathers' definition

A test is not a unit test if:

• It talks to the database
• It communicates across the network
• It touches the file system
• It can't run at the same time as any of your other unit tests
• You have to do special things to your environment (such as editing config files) to run it.

A test is not a unit test if it is not testing a unit.

Seriously, that's all there is to it.

The concept of "unit" in unit testing is not well-defined, in fact, the best definition I have found so far, isn't actually a definition because it is circular: a unit in a unit test is the smallest possible thing that can be tested in isolation.

This gives you two checkpoints: is it tested in isolation? And is it the smallest possible thing?

Please note that both of these are context-dependent. What might be the smallest possible thing in one situation (say, an entire object) might in another situation just one small piece of one single method. And what counts as isolation in one situation might be in another (e.g. in a memory-managed language, you never run in isolation from the garbage collector, and most of the time that is irrelevant, but sometimes it might not be).

Difficult one...

For me a unit test verifies one specific piece of logic in isolation. Meaning, I take some logic, extract it from the rest (if necessary by mocking dependencies) and test just that logic - a unit (of the whole) - by exploring different kind of possible control flows.

But on the other side...can we always 100% say correct or incorrect?? Not to become philosophical, but - as also Michael says in his post:

Tests that do these things aren't bad. Often they are worth writing, and they can be written in a unit test harness. However, it is important to be able to separate them from true unit tests so that we can keep a set of tests that we can run fast whenever we make our changes.

So why shouldn't I write a unit test that verifies the logic of parsing for instance an xls file by accessing some dummy file from the file system in my test folder (like MS tests allow with the DeploymentItem)?

Of course - as mentioned - we should separate these kind of tests from the others (maybe in a separate test suite in JUnit). But I think one should also write those tests if he feels comfortable in having them there...clearly then always again remembering that a unit test should just test a piece in isolation.

What is most important in my eyes is that these tests run fast and don't take too long s.t. they can be run repeatedly and very often.

It has no asserts, and is not expecting an exception to be thrown.

A test is not an Unit Test when:

• it tests more than one thing at once (i.e. it tests how two things work together) - then it is an integration test

Checklist for good unit tests:

• they are automated
• they are repeatable
• they are easy to implement
• they remain for future use, once written
• they can be run by anyone
• they can be run by the push of a button
• they run quickly

Some more best practices (in no particular order of importance):

• tests should be separated from integration tests (which are slower), so that they can be run fast as frequently as possible
• they should not comprise too much logic (preferably, no control structures)
• every test should test only one thing (thus, they should contain only one assert)
• the expected values used in assertions should be hard-coded and not computed at test run-time
• external dependencies (filesystem, time, memory etc.) should be replaced by stubs
• test should recreate the initial state at test shutdown
• in assertions, it is better to use a "contains..." policy, rather than "is strictly equal..." policy (i.e. we expect certain values in a collection, certain characters in a string etc.)

This is part of the knowledge I have extracted from Roy Osherove's book - The Art of Unit Testing

Implementing a test across multiple possibly failing units would not be a unit test.

Intricate question.

Say I am to program some business logic and all business logic needs to get to the data via some form of DAL.

Say that for the purposes of testing, I mock the DAL units (by creating "mockingbirds").

But those mockingbirds are of course, additional units in their own right. So even when using mocks, it might seem like I'm still bound to violate the idea of "no other units involved" when I want to unit-test my business logic module.

Of course, it is generally known that "creating mockingbirds for the DAL" can invalidate your very test itself on the count that your mockingbird deviates in some particular aspect from the DAL.

Conclusion : it is outright impossible to do "genuine unit-tests" on business modules that depend in any way on any kind of DAL, question mark ?

Corrolary : the only thing that can possible be ("genuinely" !) unit-tested is the DAL itself, question mark ?

Corrolary of the corrolary : given that the "DAL" is usually either an ORM or the very DML of some DBMS, and given that those products are usually bought as being "proven technology", what is the added value of doing any unit tests what so ever, question mark ?