How to parametrize Maven surefire plugin so I can choose which TestNG suites to run

Question

I've got many test suites in TestNG. These are XML files. I want to be able to choose multiple XML suites when running integration-test from maven.

Currently I can add the suite files to pom.xml like this:

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <configuration>
    <suiteXmlFiles>
      <suiteXmlFile>${pathToMySuiteFile_1}</suiteXmlFile>
      <suiteXmlFile>${pathToMySuiteFile_1}</suiteXmlFile>
    </suiteXmlFiles>
  </configuration>
</plugin>

This solution has some limitations. I can only change a path to the test suite I've got defined in pom.xml. So in my example it always has to be two files. I'm not able to run, lets say, 5 suites or just one.

Is there a way to somehow parametrize the whole section "suiteXmlFiles" in pom.xml ?

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <configuration>
    <suiteXmlFiles>
      ${multiple_paths_ToMySuiteFiles}
    </suiteXmlFiles>
  </configuration>
</plugin>

Running everything that matches given test group is not an option for me: I don't want to load all the suites I've got and then run just the selected tests using groups in TestNG suite. The reason being that a report that gets generated after running all the test suites with group filters is different from a report when just the selected test suites were run.

Answer 1

We also hit this issue with our tests. The current workaround that we use now is to define a property variable in the property section and inject it in sure-fire suiteXmlFiles block.

<properties>
    <!-- Default suites -->
    <batsSuiteFile>${project.build.testOutputDirectory}/BatsTests.xml</batsSuiteFile>
    <smokeSuiteFile>${project.build.testOutputDirectory}/SmokeTests.xml</smokeSuiteFile>

    <!-- Default suite files if not being specified from mvn command line -->
    <defaultSuiteFiles>${batsSuiteFile},${smokeSuiteFile}</defaultSuiteFiles>
    <suiteFile>${defaultSuiteFiles}</suiteFile>
</properties>

Then in the plugin section...

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.12</version>
    <configuration>
       <suiteXmlFiles>
          <!-- Suite file injection parameter from command line -->
          <suiteXmlFile>${suiteFile}</suiteXmlFile>
       </suiteXmlFiles>
    </configuration>
</plugin>

If nothing is specified from the command line it will fall back and default to the 2 suites specified above in the properties section. Then if you want to kick off a specified set of suite files you can do:

mvn test -DsuiteFile=test1.xml,test2.xml

Surprisingly from the maven docs you expect that the arg suiteXmlFiles should just override this from the command line and should just accept a comma delimited list of testng xmls http://maven.apache.org/plugins/maven-surefire-plugin/test-mojo.html#suiteXmlFiles

If anyone has any other better way please share.

Answer 2

According to User Property of suiteXmlFiles You can use:

mvn test -Dsurefire.suiteXmlFiles=test1.xml,test2.xml