How to detect API modifications when mocking e2e tests?

Question

I'd like to setup a solid e2e testing foundation on our team's project but I can't find a simple solution to that question :

When you are mocking all of your calls, what is the best way to detect if the actual model of the objects returned by your server has been modified ?

Your tests would still pass because they're testing an outdated version of the model but the app is potentially broken.

For example, if a mock assumes that /api/users/1 returns null if the user doesn't exist, when it actually returns an empty object, then although the tests may pass, the behaviour being tested relies on incorrect assumptions and may therefore fail in unexpected ways.

Or maybe the backend is somehow providing static json files with the latest up-to-date model and the frontend relies on this ?

This of course supposes that the people working on the backend and the people working on the frontend are separate teams.

I am using Angular 1.x and Protractor here but this doesn't really depend on the technology.

Answer 1

I think what are you doing (frontend isolation during tests) is right, keep it this way.

What you can do to verify your mocks is one of those:

1) If frontend and backend are tightly coupled and developed together - add a set of unit tests for the backend to verify API responses. This way if something changes in API, the backend tests will fail and you'll know that frontend mocks should be updated as well.

During the development you can run both sets of tests (e2e and backend unit tests) periodically or even on each code change.

2) If frontend is more or less independent from the backend, then you need to have some integration tests, which you will run besides the e2e tests. These should perform actual HTTP requests to the backend and compare returned data structure with your mocks. This way you can detect the situation when mocks become outdated.

The second approach is more reliable, but integration tests will probably be slower than backend unit tests, so you can run them automatically only on the CI server and not during the local development.

Answer 2

A similar solution to @Lablanc Meneses would be saving the response of every call in a JSON file. Save your response inside intercepter for your e2e tests and then then use the saved JSON file for e2e tests. These calls JSON would be automatically updated whenever running your api service. this would require running all your services once to store the model for the first time and then use it for your e2e tests. Model will be updated as soon as you run your updated api. you can create a toggle to stop saving response for you production build.

Answer 3

You need to register an http interceptor that stores the data on request: window.e2eHttp[request.url] = null;, response+responseError: window.e2eHttp[request.url] = result;

You can use protractor's angular module system to inject or use a flag in your solution, e2eService.isEnabled(), to toggle on and off the interceptor.

Then in your e2e test you need to implement a browser.wait + browser.executeScript until window.e2eHttp[request.url] has data. It will always have angular http response object (http status headers, data, ect)