Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Strategies for Class/Schema aware test data generation for Data Driven Tests

I've recently started pushing for TDD where I work. So far things are going well. We're writing tests, we're having them run automatically on commit, and we're always looking to improve our process and tools.

One thing I've identified that could be improved is how we set up our Test Data. In strictly unit tests, we often find ourselves instantiating and populating complex CLR objects. This is a pain, and typically the test is then only run on a handful of cases.

What I'd like to push for is Data Driven tests. I think we should be able to load our test data from files or maybe even generate them on the fly from a schema (though I would only consider doing it on the fly if I could generate every possible configuration of an object, and that number of configurations was small). And there is my problem.

I have yet to find a good strategy for generating test data for C# CLR objects.

I looked into generating XML data from XSDs and then loading that into my tests using the DataSourceAttribute. The seemed like a good approach, but I ran into troubles generating XSD files. xsd.exe falls over because our classes have interface members. I also tried using svcutil.exe on our assembly, but because our code is monolithic the output is huge and tricky (many interdependent .xsd files).

What are other techniques for generating test data? Ideally the generator would follow a schema (maybe an xsd, but preferably the class itself), and could be scripted. Technical notes (not sure if this is even relevant, but it can't hurt):

  • We're using Visual Studio's unit testing framework (defined in Microsoft.VisualStudio.TestTools.UnitTesting).
  • We're using RhinoMocks

Thanks

Extra Info

One reason I'm interested in this is to test an Adapter class we have. It takes a complex and convoluted legacy Entity and converts it to a DTO. The legacy Entity is a total mess of spaghetti and can not be easily split up into logical sub-units defined by interfaces (as suggested). That would be a nice approach, but we don't have that luxury.

I would like to be able to generate a large number of configurations of this legacy Entity and run them through the adapter. The larger the number of configurations, the more likely my test will fail when the next developer (oblivious to 90% of the application) changes the schema of the legacy Entity.

UPDATE

Just to clarify, I am not looking to generate random data for each execution of my tests. I want to be able to generate data to cover multiple configurations of complex objects. I want to generate this data offline and store it as static input for my tests.

I just reread my question and noticed that I had in fact originally ask for random on the fly generation. I'm surprised I ask for that! I've updated the question to fix that. Sorry about the confusion.

like image 779
MetaFight Avatar asked Dec 03 '13 09:12

MetaFight


People also ask

What is a best practice while dealing with test data?

Always try to generate prerequisite data within automation, so that its unique for each automaton run. This way, you won't face issues that come with reusing the same data again and again. Also, if the application is based on a service-oriented architecture, consider generating pre-requisite data using web services.

Which of the following is used for test data generation?

Datatect is a SQL data generator by Banner Software, provides a range of realistic test data in ASCII flat files or directly generates test data for RDBMS like Oracle, Sybase, SQL Server, and Informix.

How is data-driven testing performed?

Quite simply, data-driven testing is taking a test, parameterizing it and then running that test with varying data. This allows you to run the same test case with many varying inputs, therefore increasing coverage from a single test.


1 Answers

What you need is a tool such as NBuilder (http://code.google.com/p/nbuilder).

This allows you to describe objects, then generate them. This is great for unit testing.

Here is a very simple example (but you can make it as complex as you want):

var products = Builder<Product>
                   .CreateListOfSize(10)
                   .All().With(x => x.Title = "some title")
                   .And(x => x.AnyProperty = RandomlyGeneratedValue())
                   .And(x => x.AnyOtherProperty = OtherRandomlyGeneratedValue())
                   .Build();
like image 68
Roy Dictus Avatar answered Oct 12 '22 05:10

Roy Dictus