"Heavy" simultaneous users Nginx - Laravel - Google compute engine

Question

I'm running a server in nginx with Laravel (medium static web) and I'm doing for example 500 constant load simultaneous users during 1 minute (not distributed users during that minute).

And getting this error:

unix:/var/run/php/php7.1-fpm.sock failed - Resource temporarily unavailable

cginx.conf

worker_processes auto;

events {
    use epoll;
    worker_connections 1524; #in my case it should be 1024, but well..
    multi_accept on;
}
http {
    #with this I reduce disk usage a lot
    client_body_buffer_size 10K;
    client_header_buffer_size 1k;
    large_client_header_buffers 2 1k;
    reset_timedout_connection on;

    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;

www.conf

pm.max_children = 500
pm.start_servers = 20
pm.min_spare_servers = 20
pm.max_spare_servers = 64

Results with Google compute engine:

f1-micro (1 vCPU, 0,6 GB) - Is supporting 40 - 60 requests per second
g1-small (1 vCPU, 1,7 GB) - Is maintaining 80 request per second
n1-standard (1vCPU, 3,75 GB) - - Is maintaining 130 request per second
n1-standard-2 (2vCPU, 7,5 GB) - Is maintaining 250 request per second
.
.
n1-standard-16 (16 vCPU, 60 GB) - Is maintaining 840 request per second

The last one is the first passing the test, the rest are dropping Bad Gateways errors from 200 users to 400

If I test for example not 2.000 users distributed in 30 secs with the micro instance then is fine, but not simultaneous sending requests.

Starting with 2 cores, CPUs level show perfectly fine, same as disk operations etc..

So after a loooot of tests I have some questions:

1) Is this normal? Not for me, is not normal to need 16 cores to run a simple web.. or the stress test is too heavy and it's normal?

2) Then, am I missing something? Is Google limiting request per second somehow?

3) What would be normal parameters for the given config files?

Any other help is more than welcome

Answer 1

On a machine with 2vcpu and 7gb ram i can handle more 1000 request/second You didn't mentioned the ram per request do you need, also i suggest change php socket to tcp connection, it allow me to process 10x requests

Answer 2

TBH, it is not entirely clear what you're trying to achieve with this test, especially with bringing GCE into the equation.

If your "medium static web" site is doing a dozen SQL queries for each page, possibly with a few JOINs each, as well as various other resource intensive operations, then it is hardly a surprise that you're very far away from achieving C10K.

Your test results throughout various GCE instances look reasonably consistent, proving that it's your code that's to blame. If you want to rule out GCE as the cause of your performance issues, then it seems to be that the next logical step would be to test the performance outside of it.

It seems that you're most concerned with receiving the Bad Gateway errors on the cheaper instances, so, let's figure out why that happens.

• Your backend is only capable of processing a certain number of requests in a given amount of time, on the order of a few dozens per second on the cheapest plan.

• It is configured without a clear spec of what's supposed to happen once resources are exhausted. With the configuration at hand, you can only push 40 requests per second on the cheapest instance, yet, the configuration is set to have Laravel process 500 requests simultaneously, on 1 vCPU w/ 0.6GB of total RAM, leaving each request about 1MB of RAM, which is way on the lower scale for a "medium static web" powered by a dynamic framework, resulting in an impedance mismatch.

• It is then hardly a surprise that you're getting errors, which are clearly due to the impedance mismatch as the backpressure builds up, and the backend likely runs out of RAM trying to process the never-ending requests.

So, what is the solution?

The solution is to have a clear understanding of how many resources are required to generate each page on the backend, and subsequently limit the number of simultaneous connections to the backend from the reverse proxy to never exceed such a certain number of connections, with http://nginx.org/r/limit_req and/or http://nginx.org/r/limit_conn, as appropriate. This way, you could catch and monitor the overload conditions, and provide an appropriate error message to the user, and/or script automatic dynamic resizing of your infrastructure.

In addition to the above, another good idea is to cache results of your backend, provided that it's actually "static" content that's generated, without per-user customisation, which could then let you account for the realistic situation of when a link to your site is posted at Slashdot/Reddit/Twitter, causing a huge spike of traffic to a single "static" page, which can then be cached for the duration of the whole event. Else, if the content is not actually "static", then it's up to you to decide which way to go and which compromise to take — I'd suggest seeing if the per-request customisations are actually warranted, and whether an uncustomised version might be appropriate, especially for the Slashdot-like scenarios.