curl: how to specify target hostname for https request

Question

I have a x.example which serves traffic for both a.example and b.example. x.example has certificates for both a.example and b.example. The DNS for a.example and b.example is not yet set up.

If I add an /etc/hosts entry for a.example pointing to x.example's ip and run curl -XGET https://a.example, I get a 200.

However if I run curl --header 'Host: a.example' https://x.example, I get:

curl: (51) SSL: no alternative certificate subject name matches target host name x.example

I would think it would use a.example as the host. Maybe I'm not understanding how SNI/TLS works.

Because a.example is an HTTP header the TLS handshake doesn't have access to it yet? But the URL itself it does have access to?

Answer 1

Indeed SNI in TLS does not work like that. SNI, as everything related to TLS, happens before any kind of HTTP traffic, hence the Host header is not taken into account at that step (but will be useful later on for the webserver to know which host you are connecting too).

So to enable SNI you need a specific switch in your HTTP client to tell it to send the appropriate TLS extension during the handshake with the hostname value you need.

In case of curl, you need at least version 7.18.1 (based on https://curl.haxx.se/changes.html) and then it seems to automatically use the value provided in the Host header. It alo depends on which OpenSSL (or equivalent library on your platform) version it is linked to.

See point 1.10 of https://curl.haxx.se/docs/knownbugs.html that speaks about a bug but explains what happens:

When given a URL with a trailing dot for the host name part: "https://example.com./", libcurl will strip off the dot and use the name without a dot internally and send it dot-less in HTTP Host: headers and in the TLS SNI field.

The --connect-to option could also be useful in your case. Or --resolve as a substitute to /etc/hosts, see https://curl.haxx.se/mail/archive-2015-01/0042.html for am example, or https://makandracards.com/makandra/1613-make-an-http-request-to-a-machine-but-fake-the-hostname You can add --verbose in all cases to see in more details what is happening. See this example: https://www.claudiokuenzler.com/blog/693/curious-case-of-curl-ssl-tls-sni-http-host-header ; you will also see there how to test directly with openssl.

If you have a.example in your /etc/hosts you should just run curl with https://a.example/ and it should take care of the Host header and hence SNI (or use --resolve instead)

So to answer your question directly, replace

curl --header 'Host: a.example' https://x.example

with

curl --connect-to a.example:443:x.example:443 https://a.example

and it should work perfectly.

Answer 2

The selected answer helped me find the answer, even though it does not contain the answer. The answer in the mail/archive link Patrick Mevzek provided has the wrong port number. So even following that answer will cause it to continue to fail.

I used this container to run a debugging server to inspect the requests. I highly suggest anyone debugging this kind of issue do the same.

Here is how to address the OP's question.

# Instead of this:
# curl --header 'Host: a.example'        https://x.example

# Do:
  host=a.example
  target=x.example

  ip=$(dig +short $target | head -n1)
  curl -sv   --resolve $host:443:$ip https://$host

If you want to ignore bad certificates matches, use -svk instead of -sv

curl -svk --resolve $host:443:$ip https://$host

Note: Since you are using https, you must use 443 in the --resolve argument instead of 80 as was stated on the mail/archive

Answer 3

I had a similar need. Didn't have sudo access to update hosts file.

I use resolve parameter and also added the DNS host name as a header parameter.

--resolve <dns name>:<port>:<ip addr>

curl --request POST --resolve dns_name:443:a.b.c.d 'https://dns_name/x/y' --header 'Host: dns_name' ....

Cheers..