How TDD works when there can be millions of test cases for a production functionality?

Question

In TDD, you pick a test case and implement that test case then you write enough production code so that the test passes, refactor the codes and again you pick a new test case and the cycle continues.

The problem I have with this process is that TDD says that you write enough code only to pass the test you just wrote. What I refer to exactly is that if a method can have e.g. 1 million test cases, what can you do?! Obviously not writing 1 million test cases?!

Let me explain what I mean more clearly by the below example:

 internal static List<long> GetPrimeFactors(ulong number)         {             var result = new List<ulong>();              while (number % 2 == 0)             {                 result.Add(2);                 number = number / 2;             }              var divisor = 3;              while (divisor <= number)             {                 if (number % divisor == 0)                 {                     result.Add(divisor);                     number = number / divisor;                 }                 else                 {                     divisor += 2;                 }             }              return result;         } 

The above code returns all the prime factors of a given number. ulong has 64 bits which means it can accept values between 0 to 18,446,744,073,709,551,615!

So, How TDD works when there can be millions of test cases for a production functionality?!

I mean how many test cases suffice to be written so that I can say I used TDD to achieve this production code?

This concept in TDD which says that you should only write enough code to pass your test seems to be wrong to me as can be seen by the example above?

When enough is enough?

My own thoughts are that I only pick some test cases e.g. for Upper band, lower band and few more e.g. 5 test cases but that's not TDD, is it?

Many thanks for your thoughts on TDD for this example.

Answer 1

It's an interesting question, related to the idea of falsifiability in epistemology. With unit tests, you are not really trying to prove that the system works; you are constructing experiments which, if they fail, will prove that the system doesn't work in a way consistent with your expectations/beliefs. If your tests pass, you do not know that your system works, because you may have forgotten some edge case which is untested; what you know is that as of now, you have no reason to believe that your system is faulty.

The classical example in history of sciences is the question "are all swans white?". No matter how many different white swans you find, you can't say that the hypothesis "all swans are white" is correct. On the other hand, bring me one black swan, and I know the hypothesis is not correct.

A good TDD unit test is along these lines; if it passes, it won't tell you that everything is right, but if it fails, it tells you where your hypothesis is incorrect. In that frame, testing for every number isn't that valuable: one case should be sufficient, because if it doesn't work for that case, you know something is wrong.

Where the question is interesting though is that unlike for swans, where you can't really enumerate over every swan in the world, and all their future children and their parents, you could enumerate every single integer, which is a finite set, and verify every possible situation. Also, a program is in lots of ways closer to mathematics than to physics, and in some cases you can also truly verify whether a statement is true - but that type of verification is, in my opinion, not what TDD is going after. TDD is going after good experiments which aim at capturing possible failure cases, not at proving that something is true.