How to write Unit Tests for functions that rely on dynamic data?

Question

Lets say you have a website, that uses a function to retrieve data from the database and returns the result to be displayed/parsed/etc...

Since the data that is retrieved from the database is dynamic and could potentially change every second of the day, how do you properly write a Unit Test for this function?

Let's say the function is supposed to return an array of results. Obviously a unit test could test to see if an array is returned or not. But what happens when the contents of the array itself are incorrect due to an incorrectly written MySQL query? The size of the array could be zero or the content of the array could be incorrect. Since it relies on ever-changing data, how would the Unit Test know what is correct and what isn't? Would calls to the database from within the Unit Test itself be necessary so there is something to compare it to?

How do you properly write a Unit Test for functions that rely on dynamic data?

Answer 1

Unit tests, in their ideal form, should only test one thing. In this case, you're testing two things:

1. the logic of your function
2. the database retrieval

So I would suggest the following refactor:

1. Move the database retrieval logic into a separate function
2. Have the function you want to test call that other function
3. Mock out the function that returns data so you can unit test just the logic of your app
4. If it makes sense (if you're just relying on another library to do this, then hopefully that lib already has tests), write a unit test for the dynamic retrieval function, where you can't test specifics, but can can test the structure and reasonableness of the data returned (e.g. it has all the fields set, and is a time within 5 seconds of now).

Also, typically it's a good idea to run unit tests in a test environment where you have complete control over what's stored in the database. You don't want to run these against production data.

Answer 2

If your function does something interesting beyond pulling the data out of the database you should extract the retrieval into a different function and mock it, so you can test the rest.

This still leaves you with the task of testing the database access. You can't really do a unit test for that, because that would by definition not access any db and you could just test if it sends the sql statement you think it should, but not if the sql statement actually works.

So you need a database

You have various optiones:

1) create a fixed database for such tests which doesn't get changed by the tests.

Pro: Conceptually easy Con: hard to maintain. Tests become interdependent, because they rely on the same data. No way to test stuff that does updates, inserts or deletes (let alone DDL)

2) create a database during your test. Now you have two problems: setting up the database for the test and filling it with data.

Setting up:

1) have a database server running, with a user/schema/database for erveryone who needs to run tests (at least devs + ci-server). Schema can get created using stuff like hibernate or the scripts you use for deployment.

Works great, but drives oldfashioned DBAs crazy. The application must not depend on the schema name. You will also run into problems when you have more then one schema used by the app. This setup is fairly slow. It can help to put in on fast discs. Like RAM discs

2) Have an in memory database. Easy to start from code and fast. But in most cases it will behave the same as your production database. This is of less concern if you use something that tries to hide the difference. I often use an in memory database for the first build stage and the real thing in a second stage.

Loading the testdata

1) people tell me to use dbunit. I'm not convinced it seems to be lots of XML and hard to maintain when columns or constraints change.

2) I prefer normal application code. (Java + Hibernate) in my case, but the code that writes you data into the database in production should in many cases be suitable to write test data for your test. It helps to have a little special API that hides the details of satisfying all the foreign key and stuff: http://blog.schauderhaft.de/2011/03/13/testing-databases-with-junit-and-hibernate-part-1-one-to-rule-them/