Menu
At my work we are running a group of tests that consist of about 3,000 separate test cases. Previously we were running this entire test suite on one machine, which took about 24-72 hours to complete the entire test run. We now have created our own system for grouping and distributing the tests among about three separate machines and the tests are prioritized so that the core tests get run first for more immediate results and the extra tests run when there is an available machine.
I am curious if anyone has found a good way to distribute their tests among several machines to reduce total test time for a complete run and what tools were used to achieve that. I've done some research and it looks like TestNG is moving in this direction, but it looks like it is still under quite a bit of development.
We don't plan on rewriting any of our tests, but as we add new tests and test new products or add-ons I'd like to be able to deal with the fact that we are working with very large numbers of tests. On the other hand, if we can find a tool that would help distribute our Junit 3.x tests even in a very basic fashion, that would be helpful since we wouldn't have to maintain our own tooling to do that.
782
Distribution tests are hypothesis tests that determine whether your sample data were drawn from a population that follows a hypothesized probability distribution. Like any statistical hypothesis test, distribution tests have a null hypothesis and an alternative hypothesis.
The t-test is any statistical hypothesis test in which the test statistic follows a Student's t-distribution under the null hypothesis. A t-test is the most commonly applied when the test statistic would follow a normal distribution if the value of a scaling term in the test statistic were known.
Generally, the test statistic is calculated as the pattern in your data (i.e. the correlation between variables or difference between groups) divided by the variance in the data (i.e. the standard deviation).
A statistical distribution, or probability distribution, describes how values are distributed for a field. In other words, the statistical distribution shows which values are common and uncommon. There are many kinds of statistical distributions, including the bell-shaped normal distribution.
I've seen some people having a play with distributed JUnit. I can't particularly vouch for how effective it is, but the other teams I've seen seemed to think it was straight forward enough. Hope that helps.
77
Our build people use Mozilla Tinderbox. It seems to have some hooks for distributed testing. I'm sorry not to know the details but I thought I would at least pass on the pointer to you.
It's also nice coz you can find out immediately when a build breaks, and what checkin might have been the culprit.
http://www.mozilla.org/tinderbox.html
28
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
