Connect failed: php_network_getaddresses: getaddrinfo failed: System error

Question

Connect failed: php_network_getaddresses: getaddrinfo failed: System error

The "System error" part really throws me off.

I've been battling this error for a few months, it is very sporadic. It appears to be coming from my database connector.

Restarting php-fpm seems to alleviate the issue for ~24 hours until it starts acting up again. I had thought it was maybe hitting max children with php-fpm, but after checking php-fpm status, it is not.

I've tried to correlate the error with the syslog and nginx error log for the application, I'm running out of ideas. Any ideas on how to troubleshoot this issue?

Answer 1

after some digging, I think what our problem was is that our configuration did not have max_requests set, the children were never recycling. We did have process_idle_timeout set, but we had some scripts running on cron that were keeping the processes alive.

so if for everybody else:

// amount of requests it handles before recycling the process

pm.max_requests = 500;

// max time alive as an idle process

pm.process_idle_timeout = 10s;

Answer 2

this issue popped up for us and it was confusing because it was related to a connection to Redis

I knew that Redis was not the problem, and thought it was strange that we got the "System Error" part, as you mentioned

I dug into the php-fpm source real quick to see what threw that error and found that it was related to a DNS lookup (obviously) - but why would it be an issue if I have '127.0.0.1 localhost' in /etc/hosts?

After digging more I found it was related to file descriptors and connections that weren't persistent.

I would make sure your php-fpm children are dying gracefully, so that the connection can be recycled. I also had net.ipv4.tcp_tw_recycle=0 in /etc/sysctl.conf that seemed to be causing problems.

In short, check lsof to see who is hogging the file descriptors, and if you have not already increased them, you can likely increase them!