Testing unit tests' helper methods

Question

As I am writing tests, some of them have a lot of logic in them. Most of this logic could easily be unit tested, which would provide a higher level of trust in the tests.

I can see a way to do this, which would be to create a class TestHelpers, to put in /classes, and write tests for TestHelpers along with the regular tests.

I could not find any opinion on such a practice on the web, probably because the keywords to the problem are tricky ("tests for tests").

I am wondering whether this sounds like good practice, whether people have already done this, whether there is any advice on that, whether this points to bad design, or something of the sort.

I am running into this while doing characterization tests. I know there are some frameworks for it, but I am writing it on my own, because it's not that complicated, and it gives me more clarity. Also, I can imagine that one can easily run into the same issue with unit tests.

To give an example, at some point I am testing a function that connects to Twitter's API service and retrieves some data. In order to test that the data is correct, I need to test whether it's a json encoded string, whether the structure matches twitter's data structure, whether each value has the correct type, etc. The function that does all these checks with the retrieved data would typically be interesting to test on its own.

Any idea or opinion on this practice ?

Answer 1

One of the aphorisms about TDD is that "the tests test the code, and the code tests the tests." That is, because of the red-green-refactor cycle, you see the code fail the test and then after making it work, you see it pass the test - and that alone is enough to give you pretty good confidence that the test (and all the test utility code it calls) works correctly. For characterization tests, you don't have this red-green-refactor cycle, so it may be of value for you to write tests for your test utility methods.

Answer 2

I think it's too much to test tests themselves. alfasin right, who would test tests for tests then? And it is no coincidence that you can't find much info on this topic. That's because it's just not a common practice. Usually, well-written tests should cover arrangement logic in tests themselves. But I understand your aspiration - how to be sure that test is "well-written"? The most dangerous thing here is to have passing test, that should normally fail(but passes due to bug in it). Having such a test is even worse than not having test at all. But to be honest, I did not have much such cases in practice. My advice is just to focus on writing good tests that cover all execution paths of your logic and I think you'll be fine :)