Is there a way to make integration tests fail quickly when middleware fails?

Question

Our test environment has a variety of integration tests that rely on middleware (CMS platform, underlying DB, Elasticsearch index).

They're automated and we manage our middleware with Docker, so we don't have issues with unreliable networks. However, sometimes our DB crashes and our test fails.

The problem is that the detection of this failure is through a litany of org.hibernate.exception.JDBCConnectionException messages. These come about via a timeout. When that happens, we end up with hundreds of tests failing with this exception, each one taking many seconds to fail. As a result, it takes an age for our tests to complete. Indeed, we generally just kill these builds manually when we realise they are done.

My question: In a Maven-driven Java testing environment, is there a way to direct the build system to watch out for specific kinds of Exceptions and kill the whole process, should they arrive (or reach some kind of threshold)?

We could watchdog our containers and kill the build process that way, but I'm hoping there's a cleaner way to do it with maven.

Answer 1

If you use TestNG instead of JUnit, there are other possibilities to define tests as dependent on other tests.

For example, like others mentioned above, you can have a method to check your database connection and declare all other tests as dependent on this method.

@Test
public void serverIsReachable() {}

@Test(dependsOnMethods = { "serverIsReachable" })
public void queryTestOne() {}

With this, if the serverIsReachable test fails, all other tests which depends on this one will be skipped and not marked as failed. Skipped methods will be reported as such in the final report, which is important since skipped methods are not necessarily failures. But since your initial test serverIsReachable failed, the build should fail completely. The positive effect is, that non of your other tests will be executed, which should fail very fast.

You could also extend this logic with groups. Let's say you're database queries are used by some domain logic tests afterwards, you can declare each database test with a group, like

@Test(groups = { "jdbc" })
public void queryTestOne() {}

and declare you domain logic tests as dependent on these tests, with

@Test(dependsOnGroups = { "jdbc.* })
public void domainTestOne() {}

TestNG will therefore guarantee the order of execution for your tests.

Hope this helps to make your tests a bit more structured. For more infos, have a look at the TestNG dependency documentation.

Answer 2

I realize this is not exactly what you are asking for, but could help none the less to speed up the build:

JUnit assumptions allow to let a test pass when an assumption fails. You could have an assumption like assumeThat(db.isReachable()) that would skip those tests when a timeout is reached.

In order to actually speed things up and to not repeat this over and over, you could put this in a @ClassRule:

A failing assumption in a @Before or @BeforeClass method will have the same effect as a failing assumption in each @Test method of the class.

Of cause you would then have to mark your build as unstable via another way, but that should be easily doable.