malformed HTTP response with docker private registry (v2) behind an nginx proxy

Question

I have setup a Docker private registry (v2) on a CentOS 7 box following their offical documentation: https://docs.docker.com/registry/deploying/

I am running docker 1.6.0 on a Fedora 21 box.

The registry is running on port 5000, and is using an SSL key signed by a trusted CA. I set a DNS record for 'docker-registry.example.com' to be the internal IP of the server. Running 'docker pull docker-registry.example.com:5000/tag/image', it works as expected.

I setup an nginx server, running nginx version: nginx/1.8.0, and setup a dns record for 'nginx-proxy.example.com' pointing to the nginx server, and setup a site. Here is the config:

server {
   listen 443 ssl;
   server_name nginx-proxy.example.com;

   add_header Docker-Distribution-Api-Version: registry/2.0 always;

   ssl on;
   ssl_certificate /etc/ssl/certs/cert.crt;
   ssl_certificate_key /etc/ssl/certs/key.key;

   proxy_set_header Host $host;
   proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
   proxy_set_header X-Real-IP $remote_addr;
   proxy_set_header X-Forwarded-Proto $scheme;
   proxy_set_header X-Original-URI $request_uri;
   proxy_set_header Docker-Distribution-Api-Version registry/2.0;

   location / {
     proxy_pass http://docker-registry.example.com:5000;
   }
}

When I try to run 'docker pull nginx-proxy.example.com/tag/image' I get the following error:

FATA[0001] Error response from daemon: v1 ping attempt failed with error: Get https://nginx-proxy.example.com/v1/_ping: malformed HTTP response "\x15\x03\x01\x00\x02\x02"

My question is twofold.

1. Why is the docker client looking for the /v1_/ping?
2. Why am I seeing the 'malformed http response'

If I run 'curl -v nginx-proxy.example.com/v2' I see:

[root@alex amerenda] $ curl -v https://nginx-proxy.example.com/v2/
* Hostname was NOT found in DNS cache
*   Trying 10.1.43.165...
* Connected to nginx-proxy.example.com (10.1.43.165) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* SSL connection using TLS_DHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate:
*   subject: CN=*.example.com,O="example, Inc.",L=New York,ST=New York,C=US
*   start date: Sep 15 00:00:00 2014 GMT
*   expire date: Sep 15 23:59:59 2015 GMT
*   common name: *.example.com
*   issuer: CN=GeoTrust SSL CA - G2,O=GeoTrust Inc.,C=US
> GET /v2/ HTTP/1.1
> User-Agent: curl/7.37.0
> Host: nginx-proxy.example.com
> Accept: */*
> \x15\x03\x01\x00\x02\x02

If I do 'curl -v docker-registry.example.com' I get a 200 OK response. So nginx has to be responsible for this. Does anyone have an idea why this is happening? It is driving me insane!

Answer 1

 proxy_pass http://docker-registry.example.com:5000;

you are passing the request with plain HTTP (i.e. no https)

\x15\x03\x01\x00\x02\x02

And you are getting a SSL response back. So it looks like you must use https:// and not http:// to access port 5000. And you even know that you are using SSL:

The registry is running on port 5000, and is using an SSL key signed by a trusted CA...

Apart from that: please use the names reserved for examples like example.com and don't use domain names in your example which don't belong to you.