"PDO exception: php_network_getaddresses: getaddrinfo failed" after changing DNS resolvers, with DB running on AWS

Question

In our Laravel 5.2 app, we have a DB setup running on AWS, using multiple read servers (and multiple DB connections).

At night, we changed the DNS resolvers for all web servers to the Google DNS (8.8.8.8 and 8.8.4.4). Then, the servers started spitting these:

PDOException: Exception 'PDOException' with message 'SQLSTATE[HY000] [2002] php_network_getaddresses: getaddrinfo failed: Name or service not known' in /home/forge/studydrive.net/releases/20170320162143/vendor/laravel/framework/src/Illuminate/Database/Connectors/Connector.php:55

The code at Connector.php:55 is simply the PDO constructor:

public function createConnection($dsn, array $config, array $options)
{
    $username = Arr::get($config, 'username');

    $password = Arr::get($config, 'password');

    try {
        $pdo = new PDO($dsn, $username, $password, $options); // <== THIS ONE
    } catch (Exception $e) {
        $pdo = $this->tryAgainIfCausedByLostConnection(
            $e, $dsn, $username, $password, $options
        );
    }

        return $pdo;
    }

The .env config for the AWS (with sesnsitive parts changed):

DB_HOST=myapp-cluster.cluster-123abc.eu-west-2.rds.amazonaws.com
DB_HOST_WRITE=myapp-cluster.cluster-123abc.eu-west-2.rds.amazonaws.com
DB_HOST_READ=myapp-cluster.cluster-ro-123abc.eu-west-2.rds.amazonaws.com

So, it seems like PHP's PDO suddenly cannot resolve the DB_HOST name.

Note, that we cannot use an IP for the DB_HOST, as the hostname (on the AWS side) maps to 3 different database READ servers.

Also, before we changed the resolvers, all was working fine.

Anyone has any idea what might be causing the PDO error, or how to resolve it?

Answer 1

This is a bug in libc, according to:

https://bugs.launchpad.net/ubuntu/+source/glibc/+bug/1674733

php_network_getaddresses: getaddrinfo failed: Name or service not known ubuntu