Unit testing - how do I test a function that returns random output?

Question

I have a function which takes in two parameters, and returns one or the other 50% of the time.

The unit test for this should determine that both parameters could be returned. Luckily, I don't need to prove that the probability for each is 50% but I do need to show that both parameters are possible to be returned.

How do I write a test case for this function?

Answer 1

If the randomness is based on a random number generator that it calls, you can rig up a stub random() function that returns various results to give you your expected outputs.

Answer 2

You can do one of two things:

1. make a stub random() generator that can return consistent numbers when unit testing.
2. run your code N times, where N is high enough for you to get consistent results so long as the code is working.

Option #1 is nearly always preferred. By including your random number generator you're no longer testing an atomic unit of code, and by including a stochastic component into unit testing, you need to do statistics to verify that the code is working correctly. There are some bugs that option #2 can never pick up in a reasonable number of tests, which is worrying.

Answer 3

Not sure if this is the best way, but I would use a while loop with two flags that I set based on the values I get from the function. I would also use some sort of limit so that the test doesn't go into an infinite loop if the test fails (i.e., doesn't received one of the two values). So:

int limit = 100;
int i = 0;
while(!firstValueReceived && !secondValueReceived && i < limit) {
   int value = randomFunc();

   if(!firstValueReceived) {
      firstValueReceived = (value == firstValue);
   }

   if(!secondValueReceived) {
      secondValueReceived = (value == secondValue);
   }
   i++;
}

assertTrue(firstValueReceived && secondValueReceived);

I guess you could use the test hanging as a metric to decide if it failed or not, so you could do away with the limit:

while(!firstValueReceived && !secondValueReceived) {
   int value = randomFunc();

   if(!firstValueReceived) {
      firstValueReceived = (value == firstValue);
   }

   if(!secondValueReceived) {
      secondValueReceived = (value == secondValue);
   }
}

assertTrue(firstValueReceived && secondValueReceived);