SNI client-side mystery using Java8

Question

I have an Apache web server that runs several TLS virtualhosts with different certs and SNI.

I can access the various virtual hosts just fine using curl (presumably SNI makes it work). I can also access them fine with a little command-line Java program that basically just openConnection()s on a URL.

In my Tomcat application, the basic same client-side code accesses the same Apache server as a client, but always ends up with the default cert (defaulthost.defaultdomain) instead of the cert of the virtual host that was specified in the URL that it attempts to access. (This produces a SunCertPathBuilderException -- basically it can't verify the certificate path to the cert, which of course is true as it is a non-official cert. But then the default cert should not be used anyway.)

It's just as if SNI had been deactivated client-side in my application / Tomcat. I am at a loss why it should behave differently between my app and the command-line; same JDK, same host etc.

I found property jsse.enableSNIExtension, but I verified that it is set to true for both cases. Questions:

1. Any ideas, even wild ones, why these two programs behave differently?

2. Any ideas how I would debug this?

This is Arch Linux on 86_64, JDK 8u77, Tomcat 8.0.32.

Answer 1

This is a Java 8 bug (JDK-8144566) fixed by 8u141. See Extended server_name (SNI Extension) not sent with jdk1.8.0 but send with jdk1.7.0 for more.

Answer 2

This answer comes late, but we just have hit the problem (I can't believe it, it seems a very big bug).

All what it said seems true, but it's not default HostnameVerifier the culprit but the troubleshooter. When HttpsClient do afterConnect first try to establish setHost (only when socket is SSLSocketImpl):

SSLSocketFactory factory = sslSocketFactory;
try {
    if (!(serverSocket instanceof SSLSocket)) {
        s = (SSLSocket)factory.createSocket(serverSocket,
                                            host, port, true);
    } else {
        s = (SSLSocket)serverSocket;
        if (s instanceof SSLSocketImpl) {
            ((SSLSocketImpl)s).setHost(host);
        }
    }
} catch (IOException ex) {
    // If we fail to connect through the tunnel, try it
    // locally, as a last resort.  If this doesn't work,
    // throw the original exception.
    try {
        s = (SSLSocket)factory.createSocket(host, port);
    } catch (IOException ignored) {
        throw ex;
    }
}

If you use a custom SSLSocketFactory without override createSocket() (the method without parameters), the createSocket well parametrized is used and all works as expected (with client sni extension). But when second way it's used (try to setHost en SSLSocketImpl) the code executed is:

// ONLY used by HttpsClient to setup the URI specified hostname
//
// Please NOTE that this method MUST be called before calling to
// SSLSocket.setSSLParameters(). Otherwise, the {@code host} parameter
// may override SNIHostName in the customized server name indication.
synchronized public void setHost(String host) {
    this.host = host;
    this.serverNames =
        Utilities.addToSNIServerNameList(this.serverNames, this.host);
}

The comments say all. You need to call setSSLParameters before client handshake. If you use default HostnameVerifier, HttpsClient will call setSSLParameters. But there is no setSSLParameters execution in the opposite way. The fix should be very easy for Oracle:

SSLParameters paramaters = s.getSSLParameters();
if (isDefaultHostnameVerifier) {
    // If the HNV is the default from HttpsURLConnection, we
    // will do the spoof checks in SSLSocket.
    paramaters.setEndpointIdentificationAlgorithm("HTTPS");

    needToCheckSpoofing = false;
}
s.setSSLParameters(paramaters);

Java 9 is working as expected in SNI. But they (Oracle) seem not to want fix this:

• https://bugs.openjdk.java.net/browse/JDK-8072464
• https://bugs.openjdk.java.net/browse/JDK-8144566
• and many more

Answer 3

After some hours of debugging the JDK, here is the unfortunate result. This works:

URLConnection c = new URL("https://example.com/").openConnection();
InputStream i = c.getInputStream();
...

This fails:

URLConnection c = new URL("https://example.com/").openConnection();
((HttpsURLConnection)c).setHostnameVerifier( new HostnameVerifier() {
        public boolean verify( String s, SSLSession sess ) {
            return false; // or true, won't matter for this
        }
});
InputStream i = c.getInputStream(); // Exception thrown here
...

Adding the setHostnameVerifier call has the consequence of disabling SNI, although the custom HostnameVerifier is never invoked.

The culprit seems to be this code in sun.net.www.protocol.https.HttpsClient:

            if (hv != null) {
                String canonicalName = hv.getClass().getCanonicalName();
                if (canonicalName != null &&
                canonicalName.equalsIgnoreCase(defaultHVCanonicalName)) {
                    isDefaultHostnameVerifier = true;
                }
            } else {
                // Unlikely to happen! As the behavior is the same as the
                // default hostname verifier, so we prefer to let the
                // SSLSocket do the spoof checks.
                isDefaultHostnameVerifier = true;
            }
            if (isDefaultHostnameVerifier) {
                // If the HNV is the default from HttpsURLConnection, we
                // will do the spoof checks in SSLSocket.
                SSLParameters paramaters = s.getSSLParameters();
                paramaters.setEndpointIdentificationAlgorithm("HTTPS");
                s.setSSLParameters(paramaters);

                needToCheckSpoofing = false;
            }

where some bright mind checks whether the configured HostnameVerifier's class is the default JDK class (which, when invoked, just returns false, like my code above) and based on that, changes the parameters for the SSL connection -- which, as a side effect, turns off SNI.

How checking the name of a class and making some logic depend on it is ever a good idea escapes me. ("Mom! We don't need virtual methods, we can just check the class name and dispatch on that!") But worse, what in the world does SNI have to do with the HostnameVerifier in the first place?

Perhaps the workaround is to use a custom HostnameVerifier with the same name, but different capitalization, because that same bright mind also decided to do case-insensitive name comparison.

'nuff said.