how many requests a node http server can handle per second without queueing any requests?

Question

Does anyone know how many requests a basic single instance of node http server can handle per second without queueing any requests?

Actually, i need to write a nodejs app, that should be able to respond to about thousand incoming requests within 100ms consistently. I'm trying to test it in a 4 cpus server and running 4 instances in cluster mode. but currently it only able to handle less than 1000 requests per second consistently within 100ms

Answer 1

See this answer:

• How many clients can an http-server can handle?

It's not about queuing but about how many concurrent requests Node can handle at the same time. It contains example benchmarks that you can use and tweak to your needs.

Now, about queuing: in a sense, every request is queued because only one thing can run at the same time in one Node process. So you can say that the answer to how many requests a Node http server can handle without queuing any requests is: one. Just like for Nginx for example.

Now, the time to respond is an entirely different matter and depends on many factors like what you actually do in those requests handlers.

For example a mean time per request that I got in my experiments here for 10000 concurrent connections was less than 2 milliseconds. If it does more then it can take more of course but there is no one number for all Node servers. It depends how efficient is your implementation.

Now, a big no-no for handling concurrency in Node are:

• Never use any "Sync" function
• Never use long running for and while loops
• Never do any CPU-intensive work in your process that handles the reqests