How to safely test code requiring an external web API

Question

I'm about to embark on the journey of writing a java library to wrap a web API and I'd like to write tests along the way, just to make sure everything is peachy. I've been using JUnit for quite some time now and am pretty comfortable using it along with tools like PowerMockito/Mockito. I am however concerned that tests may fail if the API is down or I am unable to reach it as I do eventually plan to run this on a CI server (travis-ci) and would like the build-test-deploy procedure to be as close to automated as possible.

I've done quite a bit of Googling and most of the questions I've found here are unfortunately about testing an API that the programmer has authored or can set up locally. I understand it is possible to replicate the basic functionality of the API with a little tinkering, although that feels more like a step backwards than it does forwards.

I'm currently drafting ideas in my head and so far this feels like a moderately reliable solution, although it'd be nice if somebody were able to verify this or offer a better one.

TestUtil.java

public static boolean isReachable() {
    try (Socket socket = new Socket("api.host.com", 80)) {
        return true;
    } catch (Exception e) {
        return false;
    }
}

TestCase.java

@BeforeClass 
public static void testReachable() {
    Assume.assumeTrue("API was not reachable, test cannot run", TestUtil.isReachable());
}

I'm assuming it with @BeforeClass simply out of paranoia.

However, this doesn't account for HTTP errors, only checking that something is listening on port 80. Would it be worth replacing with a HEAD request? Other than checking for errors, I'm honestly not sure. I'm reluctant to use HTTP without confirmation it's the best way as this library has the potential to get quite large.

Edit: I've just stumbled upon InetAddress#isReachable(), although according to an article I'm reading it's not the most reliable.

Answer 1

You should distinguish between unit tests and integration tests.

Unit tests should never depend on infrastructure like network and file system. All those aspects should be refactored out of the way, e.g. in a separate class or method which is mocked during the unit test. I always write unit tests as 'white box' tests, where I try to cover each possible flow in the code by using a code coverage tool.

In your case you could write unit tests for the business logic in your project, like which API calls to make in which order, logic rules depending on the results of the API calls, maybe some content-related validation and error handling, mapping of your domain objects to the remote API's protocol etc.

This leaves just the parts that actually call the API untested. To do this I would run an embedded web server (e.g. Jetty) which hosts a mock version of the remote API offering cooked responses. Then you can write integration tests that call this local server to check your network code and configuration thereof.

I have often skipped the integration test part when I was using a framework like Spring-WS or JAXB because it means doing a lot of work to just test your configuration, there is no real need to test the code of such frameworks. This might just me being lazy but I always try to weigh the effort of creating tests against the expected benefit.

It becomes a different story when you have a complex landscape of services with a lot of compex mapping, configuration and routing going on. Then your integration tests are the best way to validate that all your services are wired up and talking to each other correctly. I would call these 'black box' tests, where you specify tests in terms of expected functionality of the system as a whole (e.g. user stories), independent of implementation details.

Answer 2

Depending on the size of the API and what you intend on doing with it, the code (as you said yourself) might become quite large. Before embarking on your journey, perhaps you could evaluate something that already exists to do this like RetroFit.

It's also worth mentioning that RetroFit by the nature of what it does talks to external APIs. It also has tests and is open source. You could look at their tests for inspiration.