Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Integration testing: Mock external API vs. use external API sandbox

We're required to use the API of an external partner. The API is in a good shape and we got access to a sandbox environment we can use for automatic testing.

We already test every single call of the external API using unit tests but are unsure regarding best practices for integration tests when it comes to complex operations on the side of the external partner.

Example: Every user of our service also got a user object at our external partner. When performing external API call X on this user object, we expect object Y to appear inside collection Z of this user (which we have to query using a different call).

What are best practices for testing cases like this?

  • Mock the external API as much as possible and rely on the unit tests to do their job? Advantages: Tests run fast and independent from an internet connection. Disadvantages: Mistakes is in our mocks could lead to false positives.

  • Integrate the external API sandbox and run every integration test against it. Advantages: Close to real life API interactions. Disadvantages: Tests can only be run with an open internet connection and take more time.

  • Use a hybrid of mocked and sandbox data, set a boolean to switch between the internal (=mocked) and external (=sandbox) environment when required. Advantages: Reliable tests. Disadvantages: Could be a pain to set up.

  • Other best practices?

Thanks!


Related: How are integration tests written for interacting with external API? However, the answer "You don't. You have to actually trust that the actual API actually works." is not sufficient in our opinion.

[EDIT] We fear that integration testing only against our assumptions how the external API should work (even if they are based on unit tests) – and not against the actual API – will leave us with false positives. What we'd need is a test that verifies that our assumptions (mocks) are actually correct – not only in the context of unit tests but also in the context of complex operations with several steps.

Validation might be a good example: What if we mess up the integration code and send malformed data or data that does not make any sense in the context we send it in because we missed a step? Our mock API, which does not validate (or only in very limited range) would still return valid data instead of passing the error we would receive from the real API.

like image 379
zerodot Avatar asked Jan 21 '15 14:01

zerodot


People also ask

What is the difference between API testing and integration testing?

Integration is just about facilitating interaction between two machines or systems over a network, whereas API acts as an interface between two applications.

What is API mock test?

What is Mock API? Mock APIs are used to simulate actual APIs where you can generate requests with custom data and get realistic responses the actual API would return. It is a fast and easy way of testing your REST API even before it goes live.

Is @springboottest integration test?

Spring boot provides @SpringBootTest annotation which starts the embedded server, creates a web environment and then enables @Test methods to do integration testing. Use it's webEnvironment attribute for it. It also creates the ApplicationContext used in our tests.

How do you perform integration testing with @springboottest for a Web application?

As explained above, for integrating testing of a spring-boot application, we need to use @SpringBootTest. spring-boot also does provide other classes like TestRestTemplate to test the REST APIs. Like RestTemplate class, it also does have methods getForObject(), postForObject(), exchange(), etc..


1 Answers

I believe there should be 2 level of verifications we need to do when we interface with an external API:

  • API verification: verify that the API works according to its specs and/or our understanding
  • App functionality verification: verify that our business logic works according to the expectation to the API that passes API verification

In our case, we use a mock API together with real and mock API verification.

  • Mock API allows us to isolate any runtime errors/exceptions to app functionality only, so we don't blame any external party for issues
  • The same API verification is executed against both real and mock APIs, to make sure that the real one works the way we expect, as well as the mock one should mimic the real one correctly

If along the way, external API changes, API verification may turn red, triggering changes in mock API. Changes in mock API may make app verification turn red, triggering changes in app implementation. This way you never miss any gap between external API and app implementation (ideally).

Another extra benefit of having a mock API + API verification is that your developers can use it as a documentation/specification of how the API is supposed to work.

like image 193
hidro Avatar answered Sep 21 '22 10:09

hidro