-Dmaven.test.failure.ignore=true set, but build still errors on test failure

Question

I have two different freestyle Jenkins jobs that are kicked off like this (build commands lifted from the job logs):

mvn -Dcloudbees.private.release.repository.off=true -Dcloudbees.private.snapshot.plugin.repository.off=true -Dcloudbees.private.release.plugin.repository.off=true -Dcloudbees.central.repository.off=true -Dcloudbees.private.snapshot.repository.off=true clean org.jacoco:jacoco-maven-plugin:prepare-agent install -Dmaven.test.failure.ignore=true -V -Dcheckstyle.skip=true -Dpmd.skip=true -Dgpg.skip=true -B -e

and

mvn clean install cobertura:cobertura -DallTests -Dcobertura.report.format=xml -Dmaven.test.failure.ignore=true -V -Dcheckstyle.skip=true -Dpmd.skip=true

They end like this (respectively):

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12:test (default-test) on project gora-core: There are test failures.

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.17:test (default-test) on project wildfly-domain-management: Execution default-test of goal org.apache.maven.plugins:maven-surefire-plugin:2.17:test failed:

All my searching tells me that -Dmaven.test.failure.ignore=true should do the trick, so I'm stymied.

Note that I don't have write access to these projects; I'm simply trying to compile in preparation for SonarQube analysis.

UPDATE1

I turned out to have 3 jobs (and counting) with these symptoms. For the record, they are:

1. ActiveMQ (not previously listed)
2. Wildfly Core
3. Gora

The addition of -B to the ActiveMQ job (the others already had it) moved it forward to the point of an actual compile failure. Unfortunately, the job still hasn't succeeded, so I can't be sure that -B actually fixed the job.

I tried Gora locally with the same result as on Jenkins.

UPDATE2

I've since found <testFailureIgnore>false</testFailureIgnore> in the main Gora pom, but grep tells me testFailureIgnore is nowhere in the Wildfly Core project.

UPDATE3

The Wildfly Core failure can apparently be chalked up to a bug in Surefire 2.17

Answer 1

Try running Maven in debug mode (-X) to see what Maven thinks that property is before it starts actually running the tests. If someone or something has hardcoded the value in the POM (e.g. <testFailureIgnore>false</testFailureIgnore>) then it could be ignoring the command line value you're providing.

Also make sure the Jenkins job is not a Maven job type. Stephen Connolly wrote a blog post about why the Jenkins Maven job type is evil and he explicitly mentioned that the test ignore toggle is one item modified by the plugin under the covers. One of the comments on that post is:

sorry testFailureIgnore is false not skip is true..., e.g.

<configuration> <testFailureIgnore>false</testFailureIgnore> <configuration>

which will ensure that the module's tests cannot be skipped... except when the Maven job type is playing fast and loose with its auto-magic