Do you know a good tool to test API performance? [closed]

Question

I need to simulate 3G network and http requests from different geographical locations

API: HTTP restful - JSON

Geo Location: Europe, US

Client: Mobile device

Answer 1

It sounds like you're asking two separate questions:

• What are the performance characteristics of your API server? Most usefully expressed as: how many concurrent users can it serve before response times exceed your acceptable level?
• what is the performance experience on your client devices?

I'd encourage you to split those two concerns up, and test them independently. A device operating over 3G would struggle to generate enough load to stress a well-configured web server, and it's usually not cost efficient to commission thousands of load testing nodes around the world. Also, once the traffic arrives at your web server, it shouldn't really care whether it came from the same city, country or continent, or whether it originated from a mobile device or a PC or a load test server.

So, I would use any load testing tool you like to test the performance of your web API. Apache JMeter is free, but has a bit of a learning curve; however, it's available from several cloud providers, which allows you to take your tests and run them from different continents. Google "Jmeter cloud" for more details.

If performance is a key concern, you probably want to have a continuous testing regime, where you subject your code to performance tests ever week or so, and optimize as you go - leaving optimization to the end of the project is usually rather risky...

The next question is "okay, so I know my API server can serve 1000 concurrent requests with an average response time < 1 second" (or whatever) - how does that translate to end user experience?

It only makes sense to attack this once you are really clear that your web server isn't the bottleneck - because most of the decisions you make to optimize performance beyond this point are pretty major.

Logically, if you know that your webserver is responding at a certain level, the end user performance is affected by network latency and throughput. Latency is usually measured through the crude statistic of ping times: how long does it take a network packet to travel between client and server? It's hard or impossible to improve on ping times without revisiting your entire hosting strategy; it depends on the speed of light, the connectivity between your hosting farm and the dozens of other network segments between your client and the server.

Throughput is typically measured in bytes per second. Often, this is something you can affect - for instance, by making your API less verbose, using compression, etc.

3G devices typically have relatively poor network latency characteristics, but pretty decent throughput under normal circumstances. However, there are many circumstances which affect both latency and throughput which are entirely unpredictable - busy locations are the classic example: a football stadium full of 3G devices means individual users often have poor connectivity.

Testing this is hard. I'd split it up into testing for 3G devices, and testing geographical variation. To test the performance characteristics on a 3G device, I'd simulate the network conditions using bandwidth throttlers in front of a dedicated test suite (probably based on JMeter).

The final part of the jigsaw could be expensive - there are specialist companies who can test web performance from around the world. They have nodes around the world, where they can execute test scripts; they often write the scripts for you, and provide a web interface for you to run and measure the tests. I've used Keynote in the past, and found them to be very good. This kind of testing is expensive, so I only use it right at the end of the project, once I have excluded all other performance aspects from consideration.