Cannot reach SSL IP when in docker container over bridge. Getting SSL_ERROR_SYSCALL

Question

I'm having issues connecting to any server over IP + TLS but only from within a docker container when running in (the default) network bridge. I am always getting OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to W.X.Y.Z. I've tried tcpdump (in the container) and wireshark (locally on the host) to no avail.

My work partner has the same OS/Docker version and cannot reproduce the issue. I'm at a loss as to how to debug this issue.

I have tried:

• various images (ubuntu and alpine)
• various clients (curl and wget)
• various TLS versions (1.3 and 1.2)

My container:

FROM ubuntu:latest
RUN apt update && apt upgrade -y && apt install -y curl tcpdump openssl wget

The issue:

docker run -it --rm repro /bin/bash

# in the docker bash shell, if I try to curl a regular https hostname, all is well:
root@ba6f8aab182d:/# curl -v  https://www.google.com
*   Trying 172.217.10.36:443...
* TCP_NODELAY set
* Connected to www.google.com (172.217.10.36) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  ...


# if I try again but this time with the ip instead of the hostname
root@ba6f8aab182d:/# curl -v https://172.217.10.36
*   Trying 172.217.10.36:443...
* TCP_NODELAY set
* Connected to 172.217.10.36 (172.217.10.36) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to 172.217.10.36:443
* Closing connection 0
curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to 172.217.10.36:443

The above 2 calls (curl HOSTNAME then curl MATCHINGIP) work just fine on the host machine.

Extra information in the ubuntu container:

root@ba6f8aab182d:/# openssl version
OpenSSL 1.1.1f  31 Mar 2020
root@ba6f8aab182d:/# curl --version
curl 7.68.0 (x86_64-pc-linux-gnu) libcurl/7.68.0 OpenSSL/1.1.1f zlib/1.2.11 brotli/1.0.7 libidn2/2.2.0 libpsl/0.21.0 (+libidn2/2.2.0) libssh/0.9.3/openssl/zlib nghttp2/1.40.0 librtmp/2.3
Release-Date: 2020-01-08
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: AsynchDNS brotli GSS-API HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL SPNEGO SSL TLS-SRP UnixSockets

Extra information from the host:

$ docker version
Client: Docker Engine - Community
 Cloud integration: 1.0.12
 Version:           20.10.5
 API version:       1.41
 Go version:        go1.13.15
 Git commit:        55c4c88
 Built:             Tue Mar  2 20:13:00 2021
 OS/Arch:           darwin/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.5
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       363e9a8
  Built:            Tue Mar  2 20:15:47 2021
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.4
  GitCommit:        05f951a3781f4f2c1911b05e61c160e9c30eaa8e
 runc:
  Version:          1.0.0-rc93
  GitCommit:        12644e614e25b05da6fd08a38ffa0cfe1903fdec
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

$ docker network inspect bridge
[
    {
        "Name": "bridge",
        "Id": "4b8797bccccd628a6280199eb5c0372cd08d521a88a29243b174718569e9cc7e",
        "Created": "2021-04-15T17:24:33.631745871Z",
        "Scope": "local",
        "Driver": "bridge",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": null,
            "Config": [
                {
                    "Subnet": "172.17.0.0/16",
                    "Gateway": "172.17.0.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {
            "ba6f8aab182daebd4f0b0dc449929585637cd46bc532f61991bfa28c40e09ceb": {
                "Name": "flamboyant_zhukovsky",
                "EndpointID": "e29407baf0eb8ac069416a8f787794b548d31f05bf9fb6c223fb58c935aff24c",
                "MacAddress": "02:42:ac:11:00:02",
                "IPv4Address": "172.17.0.2/16",
                "IPv6Address": ""
            }
        },
        "Options": {
            "com.docker.network.bridge.default_bridge": "true",
            "com.docker.network.bridge.enable_icc": "true",
            "com.docker.network.bridge.enable_ip_masquerade": "true",
            "com.docker.network.bridge.host_binding_ipv4": "0.0.0.0",
            "com.docker.network.bridge.name": "docker0",
            "com.docker.network.driver.mtu": "1500"
        },
        "Labels": {}
    }
]

EDIT

Trying openssl s_client -cipher ALL -servername 172.217.10.36:443 -connect 172.217.10.36:443 in the container, I get:

root@ba6f8aab182d:/# openssl s_client -cipher ALL -servername 172.217.10.36:443 -connect 172.217.10.36:443
CONNECTED(00000003)
write:errno=0
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 403 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)

While on the host I get:

openssl s_client -cipher ALL -servername 172.217.10.36:443 -connect 172.217.10.36:443
CONNECTED(00000003)
depth=2 OU = GlobalSign Root CA - R2, O = GlobalSign, CN = GlobalSign
verify return:1
depth=1 C = US, O = Google Trust Services, CN = GTS CA 1O1
verify return:1
depth=0 C = US, ST = California, L = Mountain View, O = Google LLC, CN = www.google.com
verify return:1
---
Certificate chain
 0 s:/C=US/ST=California/L=Mountain View/O=Google LLC/CN=www.google.com
   i:/C=US/O=Google Trust Services/CN=GTS CA 1O1
 1 s:/C=US/O=Google Trust Services/CN=GTS CA 1O1
   i:/OU=GlobalSign Root CA - R2/O=GlobalSign/CN=GlobalSign
---
Server certificate
-----BEGIN CERTIFICATE-----
[... certificate was here ...]
-----END CERTIFICATE-----
subject=/C=US/ST=California/L=Mountain View/O=Google LLC/CN=www.google.com
issuer=/C=US/O=Google Trust Services/CN=GTS CA 1O1
---
No client certificate CA names sent
Server Temp Key: ECDH, X25519, 253 bits
---
SSL handshake has read 3206 bytes and written 339 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-CHACHA20-POLY1305
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-CHACHA20-POLY1305
    Session-ID: 052A69E409C0705AEAB8A180228C3F8E91A530504EFB06BA9214365F6B99DCAC
    Session-ID-ctx:
    Master-Key: A4EAC218352BBEAF3A43AB625266304DCF495FFE8A916C638679473AD20DC01B508158B8C0AA39A97003FEC5B8ABD7EC
    TLS session ticket lifetime hint: 100800 (seconds)
    TLS session ticket:
    [... a lot of stuff here ...]
    Start Time: 1618514134
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
---

Answer 1

Seems like the issue is with Docker-For-Mac 3.3.0 and 3.3.1.

The solution was to downgrade to 3.2.2 even though the docker engine is the same.

See https://github.com/docker/for-mac/issues/5568