Unit Testing.... a data provider?

Question

Given problem:

• I like unit tests.
• I develop connectivity software to external systems that pretty much and often use a C++ library
• The return of this systems is nonndeterministic. Data is received while running, but making sure it is all correctly interpreted is hard.

How can I test this properly?

I can run a unit test that does a connect. Sadly, it will then process a life data stream. I can say I run the test for 30 or 60 seconds before disconnecting, but getting code ccoverage is impossible - I simply dont even comeclose to get all code paths EVERY ONCE PER DAY (error code paths are rarely run). I also can not really assert every result. Depending on the time of the day we talk of 20.000 data callbacks per second - all of which are not relly determined good enough to validate each of them for consistency. Mocking? Well, that would leave me testing an empty shell of myself because the code handling the events basically is the to be tested case, and in many cases we talk here of a COMPLEX c level structure - hard to have mocking frameworks that integrate from Csharp to C++

Anyone any idea? I am short on giving up using unit tests for this part of the application.

Answer 1

Unit testing is good, but it shouldn't be your only weapon against bugs. Look into the difference between unit tests and integration tests: it sounds to me like the latter is your best choice.

Also, automated tests (unit tests and integration tests) are only useful if your system's behavior isn't going to change. If you're breaking backward compatibility with every release, the automated tests of that functionality won't help you.

You may also want to see a previous discussion on how much unit testing is too much.