Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

failed (104: Connection reset by peer)

Tags:

php

nginx

We are using the php5-fpm and nginx.


    # nginx -v
    nginx version: nginx/1.1.19
    
*

    # php -v
    PHP 5.3.27-1~dotdeb.0 with Suhosin-Patch (cli) (built: Jul 25 2013 19:30:39) 
    Copyright (c) 1997-2013 The PHP Group
    Zend Engine v2.3.0, Copyright (c) 1998-2013 Zend Technologies
        with Xdebug v2.2.3, Copyright (c) 2002-2013, by Derick Rethans

*

Got the following error,


    2013/08/21 20:00:20 [error] 12740#0: *46 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 172.17.7.12, server: fe.test.local, request: "GET /search?gender=female&ageFrom=20&ageTo=32&religionId=&casteId=&countryId=&heightFrom=&heightTo=&profileImageStatus=%28y+p%29&search=Search HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "fe.test.local", referrer: "http://fe.test.local/

PHP5-FPM configuration,


    # grep ^[^\;] /etc/php5/fpm/php-fpm.conf

    [global]
    pid = /var/run/php5-fpm.pid
    error_log = /var/log/php5-fpm.log
    log_level = notice 
    process_control_timeout = 30 
    include=/etc/php5/fpm/pool.d/*.conf


    # grep ^[^\;] /etc/php5/fpm/pool.d/www.conf 
    [www]
    listen = /var/run/php5-fpm.sock
    listen.backlog = -1
    user = www-data
    group = www-data
    pm = dynamic
    pm.max_children = 10
    pm.start_servers = 4
    pm.min_spare_servers = 2
    pm.max_spare_servers = 6
    rlimit_files = 131072 
    rlimit_core = unlimited
    chdir = /

Nginx fascgi configuration part


    location ~ \.php$                                                             
      {                                                                             
      fastcgi_index index.php;                                              
      include fastcgi_params;                                               
      fastcgi_connect_timeout 120;                                           
      fastcgi_send_timeout 600;                                             
      fastcgi_read_timeout 600;                                             
      fastcgi_buffers 8 256k;                                               
      fastcgi_buffer_size 256k;                                             
      fastcgi_busy_buffers_size 256k;                                       
      fastcgi_temp_file_write_size 256k;                                    
      client_max_body_size 20M;                                             
      fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;     
      #fastcgi_pass 127.0.0.1:9000;                                          
      fastcgi_pass unix:/var/run/php5-fpm.sock;
    }               


    # strace -f -s 8000 -p 12761
    Process 12761 attached - interrupt to quit
    epoll_wait(9, {}, 1, 944)               = 0
    epoll_wait(9, {}, 1, 1000)              = 0
    epoll_wait(9, {}, 1, 1000)              = 0
    epoll_wait(9, 269f530, 1, 1000)         = -1 EINTR (Interrupted system call)
    --- SIGCHLD (Child exited) @ 0 (0) ---
    write(6, "C", 1)                        = 1
    rt_sigreturn(0x6)                       = -1 EINTR (Interrupted system call)
    epoll_wait(9, {{EPOLLIN, {u32=15427040, u64=15427040}}}, 1, 122) = 1
    read(4, "C", 1)                         = 1
    wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 127}], WNOHANG|WSTOPPED, NULL) = 12826
    write(3, "[21-Aug-2013 19:33:37] WARNING: [pool www] child 12826 exited with code 127 after 275.844033 seconds from start\n", 112) = 112
    clone(Process 12832 attached (waiting for parent)
    Process 12832 resumed (parent 12761 ready)
    child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7fd7a31cfa10) = 12832
    [pid 12832] set_robust_list(0x7fd7a31cfa20, 0x18) = 0
    [pid 12832] dup2(1, 2)                  = 2
    [pid 12832] close(3)                    = 0
    [pid 12832] dup2(7, 0)                  = 0
    [pid 12832] geteuid()                   = 0
    [pid 12832] setrlimit(RLIMIT_NOFILE, {rlim_cur=128*1024, rlim_max=128*1024}) = 0
    [pid 12832] setrlimit(RLIMIT_CORE, {rlim_cur=RLIM_INFINITY, rlim_max=RLIM_INFINITY}) = 0
    [pid 12832] chdir("/")                  = 0
    [pid 12832] setgid(33)                  = 0
    [pid 12832] open("/proc/sys/kernel/ngroups_max", O_RDONLY) = 3
    [pid 12832] read(3, "65536\n", 31)      = 6
    [pid 12832] close(3)                    = 0
    [pid 12832] open("/etc/group", O_RDONLY|O_CLOEXEC) = 3
    [pid 12832] lseek(3, 0, SEEK_CUR)       = 0
    [pid 12832] fstat(3, {st_mode=S_IFREG|0644, st_size=777, ...}) = 0
    [pid 12832] mmap(NULL, 777, PROT_READ, MAP_SHARED, 3, 0) = 0x7fd7a31c1000
    [pid 12832] lseek(3, 777, SEEK_SET)     = 777
    [pid 12832] fstat(3, {st_mode=S_IFREG|0644, st_size=777, ...}) = 0
    [pid 12832] munmap(0x7fd7a31c1000, 777) = 0
    [pid 12832] close(3)                    = 0
    [pid 12832] setgroups(1, [33])          = 0
    [pid 12832] setuid(33)                  = 0
    [pid 12832] prctl(PR_SET_DUMPABLE, 1)   = 0
    [pid 12832] close(4)                    = 0
    [pid 12832] close(6)                    = 0
    [pid 12832] rt_sigaction(SIGTERM, {SIG_DFL, [], SA_RESTORER, 0x7fd7a02234a0}, NULL, 8) = 0
    [pid 12832] rt_sigaction(SIGINT, {SIG_DFL, [], SA_RESTORER, 0x7fd7a02234a0}, NULL, 8) = 0
    [pid 12832] rt_sigaction(SIGUSR1, {SIG_DFL, [], SA_RESTORER, 0x7fd7a02234a0}, NULL, 8) = 0
    [pid 12832] rt_sigaction(SIGUSR2, {SIG_DFL, [], SA_RESTORER, 0x7fd7a02234a0}, NULL, 8) = 0
    [pid 12832] rt_sigaction(SIGCHLD, {SIG_DFL, [], SA_RESTORER, 0x7fd7a02234a0}, NULL, 8) = 0
    [pid 12832] rt_sigaction(SIGQUIT, {0x7633f0, [], SA_RESTORER|SA_RESTART, 0x7fd7a02234a0}, NULL, 8) = 0
    [pid 12832] close(7)                    = 0
    [pid 12832] accept(0,  
    [pid 12761] write(3, "[21-Aug-2013 19:33:37] NOTICE: [pool www] child 12832 started\n", 62) = 62
    [pid 12761] wait4(-1, 0x7fff07258a7c, WNOHANG|WSTOPPED, NULL) = 0
    [pid 12761] read(4, 0x7fff07258b5f, 1)  = -1 EAGAIN (Resource temporarily unavailable)
    [pid 12761] epoll_wait(9, {}, 1, 108)   = 0
    [pid 12761] epoll_wait(9, {}, 1, 1000)  = 0
    [pid 12761] epoll_wait(9, 269f530, 1, 1000) = -1 EINTR (Interrupted system call)
    [pid 12761] --- SIGCHLD (Child exited) @ 0 (0) ---
    [pid 12761] write(6, "C", 1)            = 1
    [pid 12761] rt_sigreturn(0x6)           = -1 EINTR (Interrupted system call)
    [pid 12761] epoll_wait(9, {{EPOLLIN, {u32=15427040, u64=15427040}}}, 1, 769) = 1
    [pid 12761] read(4, "C", 1)             = 1
    [pid 12761] wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 127}], WNOHANG|WSTOPPED, NULL) = 12830
    [pid 12761] write(3, "[21-Aug-2013 19:33:39] WARNING: [pool www] child 12830 exited with code 127 after 10.053632 seconds from start\n", 111) = 111
    [pid 12761] clone(Process 12833 attached
    child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7fd7a31cfa10) = 12833
    [pid 12761] write(3, "[21-Aug-2013 19:33:39] NOTICE: [pool www] child 12833 started\n", 62) = 62
    [pid 12761] wait4(-1, 0x7fff07258a7c, WNOHANG|WSTOPPED, NULL) = 0
    [pid 12761] read(4, 0x7fff07258b5f, 1)  = -1 EAGAIN (Resource temporarily unavailable)
    [pid 12761] epoll_wait(9,  
    [pid 12833] set_robust_list(0x7fd7a31cfa20, 0x18) = 0
    [pid 12833] dup2(1, 2)                  = 2
    [pid 12833] close(3)                    = 0
    [pid 12833] dup2(7, 0)                  = 0
    [pid 12833] geteuid()                   = 0
    [pid 12833] setrlimit(RLIMIT_NOFILE, {rlim_cur=128*1024, rlim_max=128*1024}) = 0
    [pid 12833] setrlimit(RLIMIT_CORE, {rlim_cur=RLIM_INFINITY, rlim_max=RLIM_INFINITY}) = 0
    [pid 12833] chdir("/")                  = 0
    [pid 12833] setgid(33)                  = 0
    [pid 12833] open("/proc/sys/kernel/ngroups_max", O_RDONLY) = 3
    [pid 12833] read(3, "65536\n", 31)      = 6
    [pid 12833] close(3)                    = 0
    [pid 12833] open("/etc/group", O_RDONLY|O_CLOEXEC) = 3
    [pid 12833] lseek(3, 0, SEEK_CUR)       = 0
    [pid 12833] fstat(3, {st_mode=S_IFREG|0644, st_size=777, ...}) = 0
    [pid 12833] mmap(NULL, 777, PROT_READ, MAP_SHARED, 3, 0) = 0x7fd7a31c1000
    [pid 12833] lseek(3, 777, SEEK_SET)     = 777
    [pid 12833] fstat(3, {st_mode=S_IFREG|0644, st_size=777, ...}) = 0
    [pid 12833] munmap(0x7fd7a31c1000, 777) = 0
    [pid 12833] close(3)                    = 0
    [pid 12833] setgroups(1, [33])          = 0
    [pid 12833] setuid(33)                  = 0
    [pid 12833] prctl(PR_SET_DUMPABLE, 1)   = 0
    [pid 12833] close(4)                    = 0
    [pid 12833] close(6)                    = 0
    [pid 12833] rt_sigaction(SIGTERM, {SIG_DFL, [], SA_RESTORER, 0x7fd7a02234a0}, NULL, 8) = 0
    [pid 12833] rt_sigaction(SIGINT, {SIG_DFL, [], SA_RESTORER, 0x7fd7a02234a0}, NULL, 8) = 0
    [pid 12833] rt_sigaction(SIGUSR1, {SIG_DFL, [], SA_RESTORER, 0x7fd7a02234a0}, NULL, 8) = 0
    [pid 12833] rt_sigaction(SIGUSR2, {SIG_DFL, [], SA_RESTORER, 0x7fd7a02234a0}, NULL, 8) = 0
    [pid 12833] rt_sigaction(SIGCHLD, {SIG_DFL, [], SA_RESTORER, 0x7fd7a02234a0}, NULL, 8) = 0
    [pid 12833] rt_sigaction(SIGQUIT, {0x7633f0, [], SA_RESTORER|SA_RESTART, 0x7fd7a02234a0}, NULL, 8) = 0
    [pid 12833] close(7)                    = 0
    [pid 12833] accept(0,  
    [pid 12761]  {}, 1, 766) = 0
    [pid 12761] epoll_wait(9, {}, 1, 1000)  = 0`enter code here`
    [pid 12761] epoll_wait(9, {}, 1, 1000)  = 0

like image 918
user1960200 Avatar asked Aug 22 '13 11:08

user1960200


People also ask

What is error 104 reset by peer?

It means that TCP reset has been sent to your computer. This happens for example when web server is restarted due to configuration change. Usually you solve this problem by reloading web page or waiting few minutes for maintanance window to close.

What is os error 104?

When the port related to the ProLE has issues with connection from the operating system level, the system error 104 is seen. This could be resulted from the port in use by some other process or application.

What does it mean when Facebook says Connection reset by peer?

An application gets a connection reset by peer error when it has an established TCP connection with a peer across the network, and that peer unexpectedly closes the connection on the far end.


1 Answers

The above nginx error indicate that issue is related to the fastcgi(php-fpm).

After enabling the "catch_workers_output" on /etc/php5/fpm/pool.d/www.conf Got the relevant error on php5-fpm.log. Now understand that issue is related to AMQP module.

[21-Aug-2013 23:32:57] WARNING: [pool www] child 16091 said into stderr: "php-fpm: pool www: symbol lookup error: /usr/lib/php5/20090626/amqp.so: undefined symbol: amqp_open_socket"

Issue has been fixed afterrecompiled and install amqp module

like image 133
user1960200 Avatar answered Nov 01 '22 22:11

user1960200