JUnit test for an expected timeout

Question

I have a test of communication over a TCP socket where I expect the server to not respond within a set time frame when I send a certain message.

The servers behaviour is nothing I can control.

I know how to fail a test if it has not completed within a set time frame. But how can I do the opposite, make it pass for not completing within the timeframe?

I can use @Test (timeout=1000) to make a test fail if not complete within a second.

But, using Junit 4, is there a function to test for an expected timeout as a positive result? I.e. The test will fail if completed within the time frame and pass if not?

Answer 1

Good question. Actually you can do this using only junit tools. My idea is to inverse Timeout rule behaviour + use expected attribute of Test annotation. The only one limitation: you have to place your test in separate class, because Rule applies to all tests inside it:

public class Q37355035 {

    private static final int MIN_TIMEOUT = 100;

    @Rule
    public Timeout timeout = new Timeout(MIN_TIMEOUT) {
        public Statement apply(Statement base, Description description) {
            return new FailOnTimeout(base, MIN_TIMEOUT) {
                @Override
                public void evaluate() throws Throwable {
                    try {
                        super.evaluate();
                        throw new TimeoutException();
                    } catch (Exception e) {}
                }
            };
        }
    };

    @Test(expected = TimeoutException.class)
    public void givesTimeout() throws InterruptedException {
        Thread.sleep(1000);
    }
}

Answer 2

I'm building upon the great answer by Andremoniy, don't forget to upvote his answer if you like this one!

I used the following modification to skip a test if it doesn't finish in the expected time frame. The benefit of this is that the test will be marked as Skipped by JUnit instead of Successful. This is good for optimistic execution of tests which can sometimes hang or not finish fast enough, but you don't want to mark them as failed or delete them.

public class OptimisticTestClass {

    private static final int TEST_METHOD_TIMEOUT_MS = 100;

    @Rule
    public Timeout timeout = new Timeout(TEST_METHOD_TIMEOUT_MS, TimeUnit.MILLISECONDS) {
        public Statement apply(Statement base, Description description) {
            return new FailOnTimeout(base, TEST_METHOD_TIMEOUT_MS) {
                @Override
                public void evaluate() throws Throwable {
                    try {
                        super.evaluate();
                    } catch (TestTimedOutException e) {
                        Assume.assumeNoException("Test did not finish in the allocated time, skipping!", e);
                    }
                }
            };
        }
    };

    // The test times out and is skipped
    public void givesTimeout() throws InterruptedException {
        Thread.sleep(1000);
    }
}