Full Integration Testing for NodeJS and the Client Side with Yeoman and Mocha

Question

I got awesome client side tests that I run with Yeoman. Yeoman compiles my CoffeeScript, opens up the test page in a server, visit it with PhantomJS and pass all the tests results to the command line. The process is pretty hacky, the test results are passed via alert() messages to the Phantom process which creates a temporary file and fills it with the messages as JSON. Yeoman (well, Grunt) loops over the temporary file, parses the tests and displays them in the command line.

The reason I explained the process is that I want to add a few things to it. I got server side tests as well. They use mocha and supertest to check the API endpoints and a Redis client to make sure the database state is as expected. But I want to merge those two test suites!

I don't want to write client side mock response for the server calls. I don't want to send the server mock data. Somewhere along the way I'll change the server or the client and the test will not fail. I want to do a real integration testing. So, whenever a test finishes in the client side I want a hook to run a relevant test on the server side (checking db state, session state, moving to a different test page).

Are there any solutions to this? Or, altenatively, where do I start hacking on Yeoman / Grunt / grunt-mocha to make this work?

I think the Phantom Handlers in grunt-mocha is a good place to start:

// Handle methods passed from PhantomJS, including Mocha hooks.
  var phantomHandlers = {
    // Mocha hooks.
    suiteStart: function(name) {
      unfinished[name] = true;
      currentModule = name;
    },
    suiteDone: function(name, failed, passed, total) {
      delete unfinished[name];
    },
    testStart: function(name) {
      currentTest = (currentModule ? currentModule + ' - ' : '') + name;
      verbose.write(currentTest + '...');
    },
    testFail: function(name, result) {
        result.testName = currentTest;
        failedAssertions.push(result);
    },
    testDone: function(title, state) {
      // Log errors if necessary, otherwise success.
      if (state == 'failed') {
        // list assertions
        if (option('verbose')) {
          log.error();
          logFailedAssertions();
        } else {
          log.write('F'.red);
        }
      } else {
        verbose.ok().or.write('.');
      }
    },
    done: function(failed, passed, total, duration) {
      var nDuration = parseFloat(duration) || 0;
      status.failed += failed;
      status.passed += passed;
      status.total += total;
      status.duration += Math.round(nDuration*100)/100;
      // Print assertion errors here, if verbose mode is disabled.
      if (!option('verbose')) {
        if (failed > 0) {
          log.writeln();
          logFailedAssertions();
        } else {
          log.ok();
        }
      }
    },
    // Error handlers.
    done_fail: function(url) {
      verbose.write('Running PhantomJS...').or.write('...');
      log.error();
      grunt.warn('PhantomJS unable to load "' + url + '" URI.', 90);
    },
    done_timeout: function() {
      log.writeln();
      grunt.warn('PhantomJS timed out, possibly due to a missing Mocha run() call.', 90);
    },

    // console.log pass-through.
    // console: console.log.bind(console),
    // Debugging messages.
    debug: log.debug.bind(log, 'phantomjs')
  };

Thanks! There will be a bounty on this.

Answer 1

I don't know about Yeoman - I haven't tried it yet - but I got the rest of the puzzle running. I believe you will figure out the rest.

Why Doing Integration Tests?

In your question you were talking about the situation when you have both client-side tests and server-side tests running with mocks. I assume that for some reason you can't get both test sets running with the same mocks. Otherwise, if you changed the mocks on client-side your server-side tests would fail because they would get the broken mock data.

What you need are the integration tests so when you run some client-side code in your headless browser your server-side code would also run. Moreover, simply running your server-side and client-side code is not enough, you also want to be able to put assertions on both sides, don't you?

Integration Tests with Node and PhantomJS

Most of the examples of integration tests that I found online either use Selenium or Zombie.js. The former is a big Java-based framework to drive real browsers while the later is a simple wrapper around jsdom. I assume you're hesitant to use either of those and would prefer PhantomJS. The tricky part, of course, is to get that running from your Node app. And I got just that.

There are two node modules to drive PhantomJS:

1. phantom
2. node-phantom

Unfortunately, both projects seem abandoned by their authors and other community members fork them and adapt to their needs. That means that both projects got forked numerous times and all forks are barely running. The API is almost non-existent. I got my tests running with one of the phantom forks (Thank you, Seb Vincent). Here's a simple app:

'use strict';
var express = require('express');

var app = express();

app.APP = {}; // we'll use it to check the state of the server in our tests

app.configure(function () {
    app.use(express.static(__dirname + '/public'));
});

app.get('/user/:name', function (req, res) {
    var data = app.APP.data = {
        name: req.params.name,
        secret: req.query.secret
    };
    res.send(data);
});

module.exports = app;

    app.listen(3000);
})();

It listens for request to /user and returns path parameter name and query parameter secret. Here's the page where I call the server:

window.APP = {};

(function () {
    'use strict';

    var name = 'Alex', secret ='Secret';
    var xhr = new XMLHttpRequest();
    xhr.open('get', '/user/' + name + '?secret=' + secret);
    xhr.onload = function (e) {
        APP.result = JSON.parse(xhr.responseText);
    };
    xhr.send();
})();

And here's a simple test:

describe('Simple user lookup', function () {
    'use strict';

    var browser, server;

    before(function (done) {
        // get our browser and server up and running
        phantom.create(function (ph) {
            ph.createPage(function (tab) {
                browser = tab;
                server = require('../app');
                server.listen(3000, function () {
                    done();
                });
            });
        });
    });

    it('should return data back', function (done) {
        browser.open('http://localhost:3000/app.html', function (status) {

            setTimeout(function () {
                browser.evaluate(function inBrowser() {
                    // this will be executed on a client-side
                    return window.APP.result;
                }, function fromBrowser(result) {
                    // server-side asserts
                    expect(server.APP.data.name).to.equal('Alex');
                    expect(server.APP.data.secret).to.equal('Secret');
                    // client-side asserts
                    expect(result.name).to.equal('Alex');
                    expect(result.secret).to.equal('Secret');
                    done();
                });
            }, 1000); // give time for xhr to run

        });
    });
});

As you can see I have to poll the server inside the timeout. That's because all the phantom bindings are incomplete and too limiting. As you can see I'm able to check both client state and server state in a single test.

Run your tests with Mocha: mocha -t 2s You'll probably need to increase the default timeout setting for more evolved tests to run.

So, as you can see the whole thing is doable. Here's the repo with complete example.