Where should the integration tests be stored when using the maven-failsafe-plugin?

Question

Do I have to place my integration tests under src/test with the rest of my unit tests and just distinguish them by a pattern such as *Integr*Test, *ITTest, or can they be in src/it (as is the case when developing Maven plugins and using the maven-invoker-plugin)?

I'm asking this because, to me it looks not clean enough if both unit and integration tests are in the same place, (even if they were to be controlled via a Maven profile).

Answer 1

You are right that src/it is intended to be used for integrations test of plugins. This is mentioned in the Standard Directory Layout.

The maven-failsafe-plugin, by default, will look for your integration tests inside ${project.build.testSourceDirectory}, which is the same as the maven-surefire-plugin for unit tests. By default, this corresponds to src/test/java. The integration tests are made distinct by following a naming convention:

<includes>
  <include>**/IT*.java</include>
  <include>**/*IT.java</include>
  <include>**/*ITCase.java</include>
</includes>

which is different than the naming convention for unit-tests:

<includes>
  <include>**/Test*.java</include>
  <include>**/*Test.java</include>
  <include>**/*TestCase.java</include>
</includes>

So while they would reside in the same source folder (src/test/java), the difference in names clearly distinguishes them. Also, this is the default set-up so no extra configuration would be needed.

That said, you can have other options:

• Place the integration tests inside a different source folder. This will require some configuration to make it work: you will need to use the build-helper-maven-plugin:add-test-source goal to add the custom folder as a test source folder.
• Use a different module (if you have a multi-module Maven project) that would only contain the integration tests.

Answer 2

First maven-fails-plugin runs by default in another life cycle phase (integration-test) as maven-surefire-plugin (test) does. Furthermore you can configurate maven-failsafe-plugin to run verify goal at the post-integration-test test phase if you like to check if integration tests have failed. This can be freely configured.

There is one question coming into my mind. You have 10 modules and now you would like to have integration tests? To which module do they belong? So best is to have a separate module cause they belong to none of the 10 modules.

Apart from that maven-surefire-plugin is already configured in default life cycle. Yes a supplemental goal would be an idea, but it would confuse user to use the same plugins in different relationships. So separation of concerns is important here. Apart from the whole default configurations...Those plugins share a larger code base but there are differences...

Also what has already been mentioned by Tunaki is pre-integration-test, for setup things like servers etc. integration-test and things like shutting down services/servers in post-integration-test phase. This will never happen in unit tests.

Using a separate module makes it usually simpler to setup the IT's which means having different dependencies (classpath) than for the unit tests. For example things like Arquillian.org which is never using in unit tests. This can't be handled in a single module...also a good things is separation of concerns here..

Furthermore integration tests could not be parallized by default whereas unit tests can be by definition otherwise they not unit tests.

So what about folder layout? In the integration test module you can simply use the src/test/java folder which means you don't need supplemental configuration etc. (for example via build-helper-maven-plugin etc.) which makes it easier and follows more the convention over configuration paradigm.

And not to forget you can better control what is running in your build (CI)..

And another important hint. Usually integration tests are often related to infrastructure so it might sometimes useful to ignore failures there which can simply handle by using check goal of maven-failsafe-plugin....

An example for an IT module can be found here.