I am using /server-status to monitor the Apache processes. When starting they look like this:
_____W_K__K____________C_K________C_____________W_..............
................................................................
................................................................
But after few hours of running look like this:
R_KCR___KR__RKRR_RRRKRRRRRRKRR_RRCK____R_RRRR_RRRKRRRKRRRRRRRRR_
R_RRRR_R.RR.R_R.R_R..CKRRRRW.K_RCRKRR_R_.._R._.RK_KRK_.RRR.KK_.R
..RR............................................................
There is too many "Read" (R) status which took a long time and I don't know what they do, because they even have no requests. (Mention that there are additional "." statuses which I skipped from the example above, in total I have 2000 positions available.) In the process list I have approx 40 "R" processes like this one:
Srv PID Acc M CPU SS Req Conn Child Slot Client VHost Request
15-2 21291 0/37/11158 R 0.03 7468 2 0.0 1.93 198.35 82.78.95.105
The report's header look like this:
Server Version: Apache/2.4.10 (Debian) mod_fcgid/2.3.9 OpenSSL/1.0.1t
Server MPM: prefork
Server Built: Sep 15 2016 20:44:43
Current Time: Thursday, 12-Jan-2017 08:38:46 EET
Restart Time: Wednesday, 11-Jan-2017 00:51:18 EET
Parent Server Config. Generation: 3
Parent Server MPM Generation: 2
Server uptime: 1 day 7 hours 47 minutes 27 seconds
Server load: 0.34 0.35 0.39
Total accesses: 1599556 - Total Traffic: 29.9 GB
CPU Usage: u18.87 s6.81 cu0 cs0 - .0224% CPU load
14 requests/sec - 274.0 kB/second - 19.6 kB/request
90 requests currently being processed, 27 idle workers
I have a private server with: 4X3.6GHZ Xeon processor, 2X512 GB SSD, 32 GB memory, Debian 3.16.36-1, Apache 2.4.10, PHP 5.6.29, mod_fcgid, php-fpm, php5-cgi, prefork.
Some of my apache2.conf settings:
Timeout 14400
KeepAlive On
MaxKeepAliveRequests 1000
KeepAliveTimeout 3
HostnameLookups Off
My mpm_prefork.conf settings:
<IfModule mpm_prefork_module>
StartServers 50
MinSpareServers 25
MaxSpareServers 50
ServerLimit 2000
MaxRequestWorkers 2000
MaxConnectionsPerChild 1000
</IfModule>
Some of my fcgid.conf settings:
FcgidMaxRequestLen 1073741824
FcgidOutputBufferSize 1073741824
FcgidMaxProcesses 200
FcgidMaxProcessesPerClass 100
FcgidMinProcessesPerClass 0
FcgidProcessLifeTime 30
FcgidConnectTimeout 30
FcgidIOTimeout 14400
FcgidBusyTimeout 14400
FcgidIdleTimeout 3
FcgidIdleScanInterval 1
I have some long runing cronjobs in the servers, this is why I need that big "14400" (=4 hours) timeout on Apache and PHP processes.
Questions:
Update 1:
A have modified the timeouts to 7200 (=2 hours), this is the minimum for me to can run my cronjobs. However I did not have problems with my cronjobs.
FcgidIOTimeout 7200
FcgidBusyTimeout 7200
Also for Apache:
Timeout 7200
Based on PHP and mod_fcgid: ap_pass_brigade failed in handle_request_ipc function I did this:
FcgidOutputBufferSize 0
Based on mod_fcgid: ap_pass_brigade failed in handle_request function I did this:
FcgidMaxRequestsPerProcess 500
After few hours the server stopped responding:
Service Unavailable: The server is temporarily unable to service your request due to maintanance downtime or capacity problems. Please try again later.
In the logs I found some erros like this two:
[fcgid:warn] (104)Connection reset by peer: mod_fcgid: ap_pass_brigade failed in handle_request_ipc function
[fcgid:warn] (32)Broken pipe: mod_fcgid: ap_pass_brigade failed in handle_request_ipc function
And a bunch of like this (maybe for each requests):
[fcgid:warn] mod_fcgid: too much processes, please increase FCGID_MAX_APPLICATION
Questions 2:
Because that is how apache works: it is started as root and then spawns 'children' that are started as a normal user. Security wise is is a very good approach since the user apache2 should not be able to brick your system.
Monitoring Apache with native toolsApache has a module called mod_status that provides a fairly in-depth look at what Apache is doing at any given time, including a summary of performance data like current CPU usage and the total number of requests made since the Apache HTTPD Server was started.
This functionality comes from a standard module called status and so you can simply disable that module to remove the /server-status endpoint. Show activity on this post. #<Location /server-status> # SetHandler server-status # Order deny,allow # Deny from all # Allow from localhost 127.0.
This answer is not useful. Show activity on this post. If you have only very few or none idle workers it means Apache is using all the processes it is allowed to use and new incoming requests have to wait for older requests to finish before they can be handled.
Problem solved.
The solution was to lower the Apache's Timeout
value to a number like 15. As I realized for running a long PHP script (even for hours) do not need to this timeout to be hight, it's enought that the PHP's max_execution_time
to be big enought.
Update
I have also upgraded to PHP 7.1 with FastCGI under PHP-FPM, and I have changed the Apache's MPM mode to event as esra-s suggested, and it runs like hell. Many thanks!
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With