Automated testing for REST Api [closed]

Question

At my work we have recently put together a couple of test suites written in Java to test some RESTful APIs we built. Our Services could invoke other RESTful APIs they depend on. We split it into two suites.

---

• Suite 1 - Testing each service in isolation
  • Mock any peer services the API depends on using restito. Other alternatives include rest-driver, wiremock and betamax.
  • Tests the service we are testing and the mocks all run in a single JVM
  • Launches the service in Jetty

I would definitely recommend doing this. It has worked really well for us. The main advantages are:

• Peer services are mocked, so you needn't perform any complicated data setup. Before each test you simply use restito to define how you want peer services to behave, just like you would with classes in unit tests with Mockito.
• You can ask the mocked peer services if they were called. You can't do these asserts as easily with real peer services.
• The suite is super fast as mocked services serve pre-canned in-memory responses. So we can get good coverage without the suite taking an age to run.
• The suite is reliable and repeatable as its isolated in it's own JVM, so no need to worry about other suites/people mucking about with an shared environment at the same time the suite is running and causing tests to fail.

---

• Suite 2 - Full End to End
  • Suite runs against a full environment deployed across several machines
  • API deployed on Tomcat in environment
  • Peer services are real 'as live' full deployments

This suite requires us to do data set up in peer services which means tests generally take more time to write. As much as possible we use REST clients to do data set up in peer services.

Tests in this suite usually take longer to write, so we put most of our coverage in Suite 1. That being said there is still clear value in this suite as our mocks in Suite 1 may not be behaving quite like the real services.

---

Frisby is a REST API testing framework built on node.js and Jasmine that makes testing API endpoints easy, fast, and fun. http://frisbyjs.com

Example:

var frisby = require('../lib/frisby');

var URL = 'http://localhost:3000/';
var URL_AUTH = 'http://username:password@localhost:3000/';

frisby.globalSetup({ // globalSetup is for ALL requests
  request: {
    headers: { 'X-Auth-Token': 'fa8426a0-8eaf-4d22-8e13-7c1b16a9370c' }
  }
});

frisby.create('GET user johndoe')
  .get(URL + '/users/3.json')
  .expectStatus(200)
  .expectJSONTypes({
    id: Number,
    username: String,
    is_admin: Boolean
  })
  .expectJSON({
    id: 3,
    username: 'johndoe',
    is_admin: false
  })
  // 'afterJSON' automatically parses response body as JSON and passes it as an argument
  .afterJSON(function(user) {
    // You can use any normal jasmine-style assertions here
    expect(1+1).toEqual(2);

    // Use data from previous result in next test
    frisby.create('Update user')
      .put(URL_AUTH + '/users/' + user.id + '.json', {tags: ['jasmine', 'bdd']})
      .expectStatus(200)
    .toss();
  })
.toss();

I collaborated with one of my coworkers to start the PyRestTest framework for this reason: https://github.com/svanoort/pyresttest

Although you can work with the tests in Python, the normal test format is in YAML.

Sample test suite for a basic REST app -- verifies that APIs respond correctly, checking HTTP status codes, though you can make it examine response bodies as well:

---
- config:
    - testset: "Tests using test app"

- test: # create entity
    - name: "Basic get"
    - url: "/api/person/"
- test: # create entity
    - name: "Get single person"
    - url: "/api/person/1/"
- test: # create entity
    - name: "Get single person"
    - url: "/api/person/1/"
    - method: 'DELETE'
- test: # create entity by PUT
    - name: "Create/update person"
    - url: "/api/person/1/"
    - method: "PUT"
    - body: '{"first_name": "Gaius","id": 1,"last_name": "Baltar","login": "gbaltar"}'
    - headers: {'Content-Type': 'application/json'}
- test: # create entity by POST
    - name: "Create person"
    - url: "/api/person/"
    - method: "POST"
    - body: '{"first_name": "Willim","last_name": "Adama","login": "theadmiral"}'
    - headers: {Content-Type: application/json}

I used SOAP UI for functional and automated testing. SOAP UI allows you to run the tests on the click of a button. There is also a spring controllers testing page created by Ted Young. I used this article to create Rest unit tests in our application.

One of the problems of doing automated testing for APIs is that many of the tools require you to have the API server up and running before you run your test suite. It can be a real advantage to have a unit testing framework that is capable of running and querying the APIs in a fully automated test environment.

An option that's good for APIs implemented with Node.JS / Express is to use mocha for automated testing. In addition to unit tests, its easy to write functional tests against the APIs, separated into different test suites. You can start up the API server automatically in the local test environment and set up a local test database. Using make, npm, and a build server, you can create a "make test" target and an incremental build that will run the entire test suite every time a piece of code is submitted to your repository. For the truly fastidious developer, it will even generate a nice HTML code-coverage report showing you which parts of your code base are covered by tests or not. If this sounds interesting, here's a blog post that provides all the technical details.

If you're not using node, then whatever the defacto unit testing framework for the language is (jUnit, cucumber/capybara, etc) - look at its support for spinning up servers in the local test environment and running the HTTP queries. If it's a large project, the effort to get automated API testing and continual integration working will pay off pretty quickly.

Hope that helps.

Runscope is a cloud based service that can monitor Web APIs using a set of tests. Tests can be , scheduled and/or run via parameterized web hooks. Tests can also be executed from data centers around the world to ensure response times are acceptable to global client base.

The free tier of Runscope supports up to 10K requests per month.

Disclaimer: I am a developer advocate for Runscope.