Unit testing advice on an object with many properties

Question

I'm fairly new to unit testing but I completely get the idea of testing individual units of code that perform a specific, testable task, however, I'm in a position where I need to write tests and provide confidence in the accuracy of an output of a method that acts on an object with over 50 properties. The combinations of the values of these properties produce an output based on rules injected from a rule definition object (using lambda expressions) which essentially equates to a percentage. These output percentages are "mission critical" and have been rather lazily tested previously, for example the quality of the rule definition class (do all the attributable percentages for each rule add up to 100%) but the actual properties of the object haven't been.

The "data" object comes from a database but I can, of course, mock it. My problem is the number of permutations of data that would need mocking and the amount of tests that would need to be written to ensure that data x,y,z (times 50 odd exponential) feels near impossible.

So, the question is, how are these situations testable in a real sense. Is scripting tests based on a known "correct" state and "correct" results even possible/sensible? Are unit tests applicable in this case and if not what alternatives are there.

By the way, this is legacy code here with a small opportunity to refactor but only if I can guarantee accuracy etc within timescales of a couple of days to do both the refactor and the tests!

Answer 1

I think you've given half of your answer yourself:

The combinations of the values of these properties produce an output based on rules injected from a rule definition object (using lambda expressions) which essentially equates to a percentage.

In your current unit test, you'd be mocking both the data and the rule. Therefore, you'd need only to ensure that the in- and output methods behave correctly.

Testing the rule is a different task. I can only guess, but usually, you'll have an Excel table, or something like that, of possible in- and output values to specify the requirements for this rule. I would convert that same table to a readable (csv) format, and use it directly to drive the rule's unit test.

Answer 2

If it is the sheer amount of combinations that is holding you back in trying to generate testcases, you could take a look at all-pair testing.

We have used PICT from microsoft to successfully minimize the amount of testcases while still having reasonable confidence to have most cases covered.

Summary

For instance, if you wish to create a test suite for partition and volume creation, the domain can be described by the following parameters: Type, Size, File system, Format method, Cluster size, and Compression. Each parameter has a limited number of possible values, each of which is determined by its nature (for example, Compression can only be On or Off) or as an equivalence partition (such as Size).

Type: Primary, Logical, Single, Span, Stripe, Mirror, RAID-5
Size: 10, 100, 500, 1000, 5000, 10000, 40000
Format method: quick, slow
File system: FAT, FAT32, NTFS
Cluster size: 512, 1024, 2048, 4096, 8192, 16384, 32768, 65536
Compression: on, off

There are over 4,700 possible combinations of these values. It would be very difficult to test all of them in a reasonable amount of time. Research shows that testing all pairs of possible values provides very good coverage and the number of test cases will remain manageable. For example, {Primary, FAT} is one pair and {10, slow} is another; a single test case can cover many pairs.

For the set of parameters shown above, PICT will produce 60 test cases.

Takeaway points

• there are over 4,700 possible combinations
• PICT will produce 60 testcases

All Pairs

The reasoning behind all-pairs testing is this: the simplest bugs in a program are generally triggered by a single input parameter.

The next simplest category of bugs consists of those dependent on interactions between pairs of parameters, which can be caught with all-pairs testing.

Bugs involving interactions between three or more parameters are progressively less common, whilst at the same time being progressively more expensive to find by exhaustive testing, which has as its limit the exhaustive testing of all possible inputs.