Is it realistic for a single server to handle 2000 HTTP requests per second?

Question

I am building a Java-based web service (using JSON as a data encoding) that will need to handle up-to 2,000 HTTP requests per second. The processing required for each request is almost negligible (a HashMap.put() method call), parsing the JSON would probably be the dominant overhead.

I am wondering whether a single High-Memory Quadruple Extra Large EC2 instance (68GB RAM, 8 cores, 64-bit) would be capable of handing as much as 2,000 HTTP requests per second?

I realize that an exact answer will be difficult, I'm just wondering whether this is within the bounds of possibility, or whether I'm smoking crack.

I'm currently using the SimpleWeb web framework, although I've noticed that it doesn't seem to be maintained currently. Can people recommend alternative embeddable HTTP servers that would be well suited to this kind of high-volume usage?

Answer 1

2000 requests per second (or 2 krps) should be well within the realm of possibility for a Java servlet, provided that you don't introduce huge bottlenecks and that the framework you are using doesn't suck too much. Given that apparently you are not accessing any backends, the task should be CPU-bound and scale very well.

The JSON serialization test of the Web Framework Benchmarks shows a lot of Java frameworks that give very good results; even with 20 database queries results are still very well over 2 krps. On Amazon they are using m1.large instances which are smaller than the ones you plan to use (c3.4xlarge, I gather).

You might try Undertow which provides a convenient servlet API and is well maintained. Netty is another possibility, although it has its own API.

Note: I realize that the question is a bit old, but the problem should still be valid.

Answer 2

This is definitely possible, in accordance to this question, Netty can handle way over 100 000 interactions per second. Some JSON parser can convert request string into JSON object or maybe you can even use a binaray variant of it, BSON, as describe here (if the messages are long or very complex). From this question looks like the number of connections that a server with recent operating system can handle is over 300 000 so much more than would be needed for your task.

Of course, this also depends on which actions do you need to take to handle the request that may be a limiting factor.