Maven failsafe plugin doesn't run parallel Test

Question

I have a Maven POM file and when I provide parallel execution options, I dont see any signs of parallel execution in the logs. And XML debugging is driving me nuts. Any thoughts what is wrong here?

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
   <modelVersion>4.0.0</modelVersion>
   <parent>
      <groupId>com.xxx.xxxx.testing.ranger</groupId>
      <artifactId>ranger-parent</artifactId>
      <relativePath>../parent/pom.xml</relativePath>
      <version>1.0.0</version>
   </parent>
   <artifactId>ranger-api-tests</artifactId>
   <name>Ranger API Tests</name>
   <description>Ranger API integration tests. Depends on the Ranger test framework project.</description>
   <properties>
      <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
      <test.groups />
      <surefire.and.failsafe.report.dir>C:/Users/xxx/Documents/Projects/LearnCubeAPIIT/tests/target</surefire.and.failsafe.report.dir>
   </properties>
   <dependencies>
      <!-- Granite Testing framework -->
      <dependency>
         <groupId>com.xxx.xxxx.testing.ranger</groupId>
         <artifactId>ranger-api-it-framework</artifactId>
         <version>1.0.0</version>
      </dependency>
      <dependency>
         <groupId>commons-lang</groupId>
         <artifactId>commons-lang</artifactId>
         <version>2.4</version>
      </dependency>
      <dependency>
         <groupId>commons-httpclient</groupId>
         <artifactId>commons-httpclient</artifactId>
         <version>3.1</version>
      </dependency>
      <dependency>
         <groupId>org.apache.maven.surefire</groupId>
         <artifactId>surefire-junit47</artifactId>
         <version>2.16</version>
      </dependency>
      <dependency>
         <groupId>junit</groupId>
         <artifactId>junit</artifactId>
         <version>4.11</version>
      </dependency>
   </dependencies>
   <build>
      <plugins>
         <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-failsafe-plugin</artifactId>
            <version>2.16</version>
            <configuration>
               <parallel>classes</parallel>
               <threadCount>4</threadCount>
               <reportsDirectory>${project.build.directory}/surefire-reports</reportsDirectory>
               <groups>${test.groups}</groups>
            </configuration>
            <executions>
               <execution>
                  <goals>
                     <goal>integration-test</goal>
                     <goal>verify</goal>
                  </goals>
               </execution>
            </executions>
         </plugin>
      </plugins>
   </build>
   <reporting>
      <plugins>
         <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-report-plugin</artifactId>
            <version>2.16</version>
            <configuration>
               <reportsDirectory>${project.build.directory}/surefire-reports</reportsDirectory>
            </configuration>
            <reportSets>
               <reportSet>
                  <id>integration-tests</id>
                  <reports>
                     <report>failsafe-report-only</report>
                     <report>report-only</report>
                  </reports>
               </reportSet>
            </reportSets>
         </plugin>
      </plugins>
   </reporting>
</project>

Logs from run - http://pastebin.com/WkCDwYzJ

Question - When parallel execution options are passed to failsafe plugin, how are the logs supposed to look like to confirm that parallel execution of tests is taking place?

UPDATE

I noticed that when I set debug option for maven to execute, I noticed that parallelExecution is being set to false

[DEBUG] Populating class realm plugin>org.apache.maven.plugins:maven-surefire-plugin:2.12.4--1057774753
[DEBUG]   Included: org.apache.maven.plugins:maven-surefire-plugin:jar:2.12.4
[DEBUG]   Included: org.apache.maven.surefire:surefire-booter:jar:2.12.4
[DEBUG]   Included: org.apache.maven.surefire:surefire-api:jar:2.12.4
[DEBUG]   Included: org.apache.maven.surefire:maven-surefire-common:jar:2.12.4
[DEBUG]   Included: org.apache.commons:commons-lang3:jar:3.1
[DEBUG]   Included: org.apache.maven.shared:maven-common-artifact-filters:jar:1.3
[DEBUG]   Included: org.codehaus.plexus:plexus-utils:jar:3.0.8
[DEBUG]   Included: org.apache.maven.reporting:maven-reporting-api:jar:2.0.9
[DEBUG]   Included: org.apache.maven.plugin-tools:maven-plugin-annotations:jar:3.1
[DEBUG]   Excluded: org.apache.maven:maven-plugin-api:jar:2.0.9
[DEBUG]   Excluded: org.apache.maven:maven-artifact:jar:2.0.9
[DEBUG]   Excluded: org.apache.maven:maven-project:jar:2.0.9
[DEBUG]   Excluded: org.apache.maven:maven-settings:jar:2.0.9
[DEBUG]   Excluded: org.apache.maven:maven-profile:jar:2.0.9
[DEBUG]   Excluded: org.apache.maven:maven-model:jar:2.0.9
[DEBUG]   Excluded: org.apache.maven:maven-artifact-manager:jar:2.0.9
[DEBUG]   Excluded: org.apache.maven:maven-plugin-registry:jar:2.0.9
[DEBUG]   Excluded: org.codehaus.plexus:plexus-container-default:jar:1.0-alpha-9-stable-1
[DEBUG]   Excluded: junit:junit:jar:3.8.1
[DEBUG]   Excluded: org.apache.maven:maven-core:jar:2.0.9
[DEBUG]   Excluded: org.apache.maven:maven-plugin-parameter-documenter:jar:2.0.9
[DEBUG]   Excluded: org.apache.maven:maven-repository-metadata:jar:2.0.9
[DEBUG]   Excluded: org.apache.maven:maven-error-diagnostics:jar:2.0.9
[DEBUG]   Excluded: org.apache.maven:maven-plugin-descriptor:jar:2.0.9
[DEBUG]   Excluded: org.apache.maven:maven-monitor:jar:2.0.9
[DEBUG]   Excluded: classworlds:classworlds:jar:1.1
[DEBUG]   Excluded: org.apache.maven:maven-toolchain:jar:2.0.9
[DEBUG] Configuring mojo org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test from plugin realm ClassRealm[plugin>org.apache.maven.plugins:maven-surefire-plugin:2.12.4--1057774753, parent: sun.misc.Launcher$AppClassLoader@7987aeca]
[DEBUG] Configuring mojo 'org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test' with basic configurator -->
[DEBUG]   (s) basedir = /Users/nallagun/Git/testing/Ranger/tests
[DEBUG]   (s) childDelegation = false
[DEBUG]   (s) classesDirectory = /Users/nallagun/Git/testing/Ranger/tests/target/classes
[DEBUG]   (s) disableXmlReport = false
[DEBUG]   (s) enableAssertions = true
[DEBUG]   (s) forkMode = once
[DEBUG]   (s) junitArtifactName = junit:junit
[DEBUG]   (s) localRepository =        id: local
      url: file:///Users/nallagun/.m2/repository/
   layout: none

[DEBUG]   (s) parallel = classes
[DEBUG]   (f) parallelMavenExecution = false
[DEBUG]   (s) perCoreThreadCount = true

Any Idea why this might be happening?

Thanks

Answer 1

Firstly, make sure you're forcing a specific jUnit provider, since older versions (<4.7) are known to have issues with concurrent execution.

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-failsafe-plugin</artifactId>
  <version>2.16</version>
  <dependencies>
    <dependency>
      <groupId>org.apache.maven.surefire</groupId>
      <artifactId>surefire-junit47</artifactId>
      <version>2.16</version>
    </dependency>
  </dependencies>
</plugin>

You can use the forkCount for concurrent execution option instead of parallel. I find it more flexible and since each test run in its own JVM, you would have less concurrency issues that might make your test results non-deterministic. These configurations apply to maven-surefire-plugin and maven-failsafe-plugin.

<forkCount>8</forkCount>
<reuseForks>true</reuseForks>

You can combine forkCount and parallel, but the different possible combinations of forkCount, reuseForks, etc make things more complicated. Note this from the docs:

When using reuseForks=true and a forkCount value larger than one, test classes are handed over to the forked process one-by-one. Thus, parallel=classes would not change anything. However, you can use parallel=methods: classes are executed in forkCount concurrent processes, each of the processes can then use threadCount threads to execute the methods of one class in parallel.

I'm not sure the reports would tell you if tests ran in concurrent mode or not, but the way I know they are running is by using top or htop (on Linux or OS X). Here's a screenshot of one my test runs. You can see multiple JVMs running tests and consuming CPU. Also my tests run much faster in this fashion.