How can I benchmark a websocket-based Node.js application?

Question

I am testing the scalability of a websockets application I wrote using socket.io and Node.js. I want to stress test and benchmark the performance. What are some tools I can use?

So far I've looked into wsbench but I'd like to explore any alternatives.

Answer 1

Spent the day looking into benchmarking node.js and socket.io, I tried a number of solutions, I found you can't use wsbench to benchmark socket.io because socket.io has its own protocol that needs to be addressed to make the connections.

After trying a few other solutions with no luck I ended up using socket.io-benchmark and I just hit 60,000 connections on a m1.large EC2 instance using only one core with room to spare. In my case I only wanted to measure the overhead of the socket.io connections (which required me to comment out some code in socket.io-benchmark) so I didn't have any messages sent between client and server.

It was interesting to see the clients took more CPU and Memory than the server. So make sure the client server you're benchmarking from is beefy, I used a EC2 c1.xlarge and had plenty of head room. And I broke the benchmark into 6 individual processes so it could scale across the multiple cores and avoid memory limits.

Be careful to set the <rampup in seconds> option high enough on the clients, if you set it to low everything buffers and grinds to halt.

The only problem with socket.io-benchmark is I had to use node v0.4.12 to get it all working due to a dependency on v8-profiler which won't compile under newer versions of node (as of writing).

Answer 2

There's a very simple Socket.IO client implementation for Node.js. It's not an actual benchmarking tool, but it does know about Socket.IO's protocol.

Using that, it's fairly easy to write a client that interacts with your application.