Benchmark Asynchronous Code (Benchmark.js, Node.js)

Question

I'd like to use the Benchmark.js module to test some asynchronous code written in node.js. Specifically, I want to fire ~10,000 requests to two servers (one written in node, one written in PHP), and track how long it takes for each server to complete all requests.

I was planning to write a simple node script to fire these requests using Benchmark, but I'm a little confused regarding how to use it with asynchronous code. Usually in node modules, there's some sort of a callback that you call when your async code is complete, or a Promise is returned from the function etc. But with Benchmark, from everything I'm reading in the docs, it doesn't seem to handle async at all.

Does anyone know what I should be doing or looking at? I can write the benchmark manually if need be; it just seems like a common enough use case that Benchmark or others would probably have already implemented it in their professional-grade testing libraries.

Thanks for any direction, ~ Nate

Answer 1

It's not very well documented, but here's a PoC:

var Benchmark = require('benchmark');
var suite     = new Benchmark.Suite();

suite.add(new Benchmark('foo', {
  // a flag to indicate the benchmark is deferred
  defer : true,

  // benchmark test function
  fn : function(deferred) {
    setTimeout(function() {
      deferred.resolve();
    }, 200);
  }
})).on('complete', function() {
  console.log(this[0].stats);
}).run();

Benchmark.js v2 slightly changes the syntax:

var Benchmark = require('benchmark');
var suite = new Benchmark.Suite;

suite.add('foo', {
  defer: true,
  fn: function (deferred) {
    setTimeout(function() {
      deferred.resolve();
    }, 200);
  }
}).on('complete', function () {
  console.log(this[0].stats)
}).run()

Answer 2

I came across this same issue when trying to test async functions. Here is an example of what I ended up using on Code Sand Box. Here is the link to the benchmark docs where it gives an example using the defer property.

And here is the code I used in Node.js for anyone who comes along and wants to see deffered used with async/await. I hope someone finds this useful!

const Benchmark = require('benchmark');
const suite = new Benchmark.Suite();
const promiseCount = 10;
const setupArray = Array.from(Array(promiseCount)).map((_, i) => i);

const sleep = (ms = 500) =>
  new Promise(resolve => {
    setTimeout(() => {
      resolve();
    }, ms);
  });

const asyncFunction = async (name, index) => {
  await sleep(100);
  return `${name}_${index}`;
};

suite
  .add("Promise.all", {
    defer: true,
    fn: async function(deferred) {
      const promiseArray = setupArray.map(asyncFunction);
      await Promise.all(promiseArray);
      deferred.resolve();
    }
  })
  .add("For loop", {
    defer: true,
    fn: async function(deferred) {
      const final = [];

      for (let i = 0; i < promiseCount; i++) {
        const data = await asyncFunction(setupArray[i], i);
        final.push(data);
      }
      deferred.resolve();
    }
  })
  .on("cycle", function(event) {
    console.log(String(event.target));
  })
  .on("complete", function() {
    console.log("Fastest is " + this.filter("fastest").map("name"));
  })
  .run({ async: true });