TestNG iterate over test data instead of test methods

Question

When using DataProvider with multiple TestNG methods, every method is run with all data sets in sequence. Instead I want to iterate over the data sets and execute all methods on each iteration. I do not care if the results show every single test method result or a summary of the runs per method.

I already tried the option

order-by-instances="true"

in a suite.xml with no success.

Sample code:

public class TestNGTest
{
    @DataProvider(name = "dp")
    public Object[][] createData(Method m) {
      return new Object[][] { new Object[] { "Cedric" }, new Object[] {"Martina"}};
    }

    @Test(dataProvider = "dp")    
    public void test1(String s) throws InterruptedException {
        System.out.println("test1 " + s);
        Thread.sleep(1000);
    }

    @Test(dataProvider = "dp")
    public void test2(String s) throws InterruptedException {
        System.out.println("test2 " + s);
        Thread.sleep(1000);
    }
}

Actual result:

test1 Cedric
test1 Martina
test2 Cedric
test2 Martina
PASSED: test1("Cedric")
PASSED: test1("Martina")
PASSED: test2("Cedric")
PASSED: test2("Martina")

Wanted result:

test1 Cedric
test2 Cedric
test1 Martina
test2 Martina
PASSED: test1("Cedric")
PASSED: test2("Cedric")
PASSED: test1("Martina")
PASSED: test2("Martina")

Answer 1

Please try with following listener GroupByInstanceEnabler. You can put this listener in Listeners annotation in your test class (or test base class if have such) or just even simpler and better solution is to put it in META-INF to let TestNg load it using ServiceLoader (http://testng.org/doc/documentation-main.html#listeners-service-loader)

This will allow you to get rid of suite.xml and only what you will need to keep this META-INF and enabler on your classpath. Anytime you will run any test this will be loaded - not need to configure anything like IDE, create suites to run - it always will load your listener out-of-the-box.

import org.testng.ISuite;
import org.testng.ISuiteListener;

public class GroupByInstanceEnabler implements ISuiteListener {

    @Override
    public void onStart(ISuite suite) {
        suite.getXmlSuite().setGroupByInstances(true);
    }

    @Override
    public void onFinish(ISuite suite) {

    }
}

Pawel