Java Https client certificate works absolutely differently than browsers or other tools

Question

It's appeared that we can't configure our java client to correctly handle SSL connection.

We can successfully connect to HTTPS URL using browser with specified client certificate and with curl tool but not with our java client or SOAP UI tool that is also java based.Even when we used the same *.p12 certificate in browser (which works fine) and in the SOAP UI tool (doesn't work).

So, We have following information:

1. Public Client certificate (pem format)
2. The client's private key (pem format)
3. CA which is used by server to validate client certificate;

We converted these pem certificates to jks and to p12 and used apache http client for connection. And as it was mentioned the same certificate is working for browser but doesn't work neither for custom client or for java based for SOAP UI.

First of all I guessed that we have following problem described there:
When the server requests a client certificate (as part of the TLS handshake), it will also provide a list of trusted CA's as part of the certificate request. When the client certificate you wish to present for authentication is not signed by one of these CA's, it won't be presented at all

But it seems if we had such problem It wouldn't resolve this certificate for browser.Because browser by default doesn't allow to use incorrect (with different issuer) certificate by default.

Client I created based on following tutorial:

So, basically I have to questions:

1. How to know whether client certificate sent at all: we used org.apache.http.conn.ssl.SSLSocketFactory probably in what place to debug;
2. What the issue might be, probably any good reference that describes differences of browsers SSL configuration and java-based tools (probably java SSL validation more strictly)?

Update 1

We tried to use openssl s_client tool. The output showed that server certificate is as we expected, but there are no CA for client certificate validation:

No client certificate CA names sent

The syntax for openssl s_client if using connection to 443 without specification of the schema (https://). So we decided to try our url in browser without the schema but with the 443 port. Server responded with:

The plain HTTP request was sent to HTTPS port

Next our step in this investigation was to try wireshark. First we got the correct session with browser and then we tried our application. We tried openssl s_client as well.

Difference is in the Server Key Exchange packet. While browser packet has a Certificate Request, but openssl and our application has no such field in that packet. Also if browser doesn't send any certificates this field is still present in the packet.

So only browser gets a request for certificate. Any thoughts about why this is happening are appreciated.

Answers to questions in comments

• We're using java 1.7
• Is it initial client cert authentication
• When we use openssl s_client -connect your.server.name:443 we get a valid server certificate back and No client certificate CA names sent. Also we get verify error:num=2:unable to get issuer certificate.
• Server certificates we get from s_client and browser are the same
• There are no list of CA certificates advertised No client certificate CA names sent

Update 2

After some investigation and research we manage to find out the root cause of the problem. So we have SNI (Server Name Indication) enabled on our server and our application is using apache httpclient 4.2.1. This version of client doesn't support the SNI extension and doesn't send the server_name extension in Client Hello packet. After that server doesn't advertise client to authenticate with client certificate. This issue was fixed in 4.3.2 release.

Right now we are trying to check if the 4.3.2 version of httpclient is correctly sending server_name extension.

Answer 1

Finally the solution for this problem was found.

So preconditions for this problem to happen are:

• Java version < 7
• Apache httpclient version < 4.3.2
• SNI is enabled on the server

Root cause of the problem is in the old version of httpclient was used. Clients before the version 4.3.2 are not capable of dealing with SNI (Server Name Indication) and don't send necessary information in Client Hello packet (field name: server_name). Here is a issue link.

Basically SNI is used to run multiple virtual domains on the same ip-address using different server certificates for each domain. When SNI enabled server is getting Client Hello packet without server_name then it can't determine what certificate it has to use and therefore is not advertising the client with CA list.

Solution is to use Java version >= 7 and apache httpclient version >= 4.3.2.
Also wireshark is really helpful for debugging such issues and the filter example to use with it are posted by juhraffe:

ssl && (ip.dst == ip_or_domain_of_server || ip.src == ip_or_domain_of_server)

Also while using it don't forget to make wireshark collect packets on correct interface.