Apache2 - Optimal values for ServerLimit, MaxClients, MaxRequestsPerChild for very HIGH traffic website

Question

I'm running a traffic intense site (100k daily). At peak time (1500+ active online) site gets drastically slowed down and increase in page loading time. ( images too )

We use front end + mysql database on same server. Have enough unutilized resources left.

load average: 1.47, 1.63, 1.73 htop: http://grabilla.com/02b13-02a8961d-bd7e-404c-9873-06e57bb7eab1.png

Server config: E3 1230 (4 x 3.2Ghz) / 16GB RAM / 1Gbps Port Speed centoOS 5.8

Apache config:

Timeout 150
KeepAlive Off
MaxKeepAliveRequests 1000
KeepAliveTimeout 15

<IfModule prefork.c>
StartServers       8
MinSpareServers    5
MaxSpareServers   20
ServerLimit      256
MaxClients       256
MaxRequestsPerChild  4000
</IfModule>

<IfModule worker.c>
StartServers         2
MaxClients         450
MinSpareThreads     25
MaxSpareThreads     75 
ThreadsPerChild     25
MaxRequestsPerChild  0
</IfModule>

CPU usage never goes beyond 2 average load on peak hour. The MySQL server also has a 7~15% usage at that time. I know it's not a DB bottleneck because static pages also take long to load on peak hours.

Any tips to optimize these values will be greatly appreciated, thanks.

website url: http://goo.gl/XVPAA

Answer 1

Assuming you use linux with prefork: turn on keep alive. If you have constant traffic increase number of servers. Lower you keep alive time so you wont saturate you connections quickly and increase you request per child so the servers wont be restarted to often. you have lot of RAM so try to maximize your ram usage and decrease you CPU usage. Try this at first:

Timeout 150
KeepAlive On
MaxKeepAliveRequests 1000
KeepAliveTimeout 5

<IfModule prefork.c>
StartServers      20
MinSpareServers   20
MaxSpareServers   30
ServerLimit      256
MaxClients       256
MaxRequestsPerChild  10000
</IfModule>

Also edit your my.conf and increase cache and buffers.