Error 502 in php-fpm without any details

Question

I am beginning to despise PHP-FPM! it's horrendous at handling errors!

I am getting a NetworkError: 502 Bad Gateway and although I know where the error is occurring because i manually un-commented line by line until i found the bad line, i don't know why that line is causing a problem.

Before you ask what the line is that is causing the error, that isn't my problem, my problem is that I can't get PHP to tell me what the error is. It just keeps responding with a 502 error.

Here is my configuration

nginx site

location ~ .+?\.php {
    fastcgi_split_path_info ^(.+?\.php)/?(.*)$;
    if (!-f $document_root$fastcgi_script_name) {
        return 404;
    }
    include fastcgi_params;
    fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    fastcgi_param PATH_INFO $fastcgi_path_info;
    fastcgi_param LOG_PATH /var/www/sites/api/logs;
    fastcgi_param ENVIRONMENT dev;
    fastcgi_buffer_size 128k;
    fastcgi_buffers 254 16k;
    fastcgi_busy_buffers_size 256k;
    fastcgi_temp_file_write_size 256k;
    proxy_intercept_errors on;
    fastcgi_intercept_errors on;
    fastcgi_pass unix:/var/run/php5-fpm.sock;
    fastcgi_index index.php;
}

php.ini

[PHP]
engine = On
expose_php = Off
max_execution_time = 30
memory_limit = 128M
default_socket_timeout = 5
session.save_path = /var/www/session/
file_uploads = Off
upload_tmp_dir = /tmp/php
upload_max_filesize = 5M
post_max_size = 5M
max_file_uploads = 1
date.timezone = 'UTC'
disable_functions = phpinfo,exec,passthru,shell_exec,system,proc_open,popen,curl_multi_exec,parse_ini_file,show_source
mail.add_x_header = Off
sql.safe_mode = On
cgi.force_redirect = 1
allow_url_fopen = Off
allow_url_include = Off

error_reporting = E_ALL
display_errors = On
display_startup_errors = On
html_errors = Off
log_errors = On
error_log = /var/log/php5-fpm.log
log_errors_max_len = 1024
ignore_repeated_errors = Off
ignore_repeated_source = Off
report_memleaks = On
track_errors = On

pool.d/www.conf

[www]
listen = /var/run/php5-fpm.sock
user = www-data
group = www-data
listen.owner = www-data
listen.group = www-data

pm = static
pm.max_children = 600
;pm.start_servers = 10
;pm.min_spare_servers = 5
;pm.max_spare_servers = 15
pm.max_requests = 100
;pm.status_path = /php_status

;request_terminate_timeout = 5s
;request_slowlog_timeout = 5s
;slowlog = /var/log/php/fpm/domain.slowlog.log

; Redirect worker stdout and stderr into main error log. If not set, stdout and
; stderr will be redirected to /dev/null according to FastCGI specs.
; Default Value: no
catch_workers_output = yes

php_flag[display_errors] = on
php_flag[display_startup_errors] = on
;php_flag[output_buffering] = off

php_admin_value[error_log] = /var/log/php5-fpm.log
php_admin_flag[log_errors] = on

and all i get in the logs is the following

site error log

2014/10/02 14:34:50 [error] 25966#0: *9 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 109.70.40.213, server: www.example.com, request: "POST /v2/payments/payment/sale HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "www.example.cm", referrer: "http://boxshop.im/checkout"

php error log

[02-Oct-2014 14:44:26.023450] DEBUG: pid 25216, fpm_event_loop(), line 419: event module triggered 2 events
[02-Oct-2014 14:44:26.023927] DEBUG: pid 25216, fpm_got_signal(), line 76: received SIGCHLD
[02-Oct-2014 14:44:26.024044] WARNING: pid 25216, fpm_children_bury(), line 252: [pool www] child 25251 exited on signal 11 (SIGSEGV) after 1441.610042 seconds from start 
[02-Oct-2014 14:44:26.025943] NOTICE: pid 25216, fpm_children_make(), line 421: [pool www] child 26039 started 
[02-Oct-2014 14:44:26.026192] DEBUG: pid 25216, fpm_event_loop(), line 419: event module triggered 1 events

a nice, Parse Error on line 234 would be nice!

frustrated

Answer 1

Process exited with singal 11 (segmentation fault). Typically, it means process unexpectedly crashed because of some error in memory usage, and php just can't handle that error in any way. You will get same error using apache+mod_php, apache+mod_fcgid or any other configuration.

In my practice, segfault gives you no useful error message in all cofigurations. The only real way to debug it is strace. One-liner to attach strace to all php processes running:

strace -f -F -s1000 -t -T `ps aux | grep -E 'apache|php|httpd' | awk '{print "-p" $2}' | xargs`

If the error is hard to reproduce, you can use xdebug php-module. You need to install it like that:

apt-get install php5-xdebug

or

yum install php-pecl-xdebug

for CentOS. For automatic tracing of all processes add to config:

xdebug.auto_trace=On

And you'll get traces, default path is:

xdebug.trace_output_dir => /tmp => /tmp
xdebug.trace_output_name => trace.%c => trace.%c

So your trace will have name /tmp/trace.$PID.xt

There you should be able to see last called fuction before process crash.