How can I use certificate authentication with HttpsURLConnection?

Question

I'm trying to connect to an HTTPS URL, but I need to use client authentication with a certificate placed on my system by third party software.

I haven't the slightest idea how I'm supposed to either find or use it and all I have to go on is C# sample code, which differs significantly with all the Java answers I've found about this. (for instance, the KeyStore needs some sort of password apparently?)

This is the C# sample code I have

System.Security.Cryptography.X509Certificates.X509CertificateCollection SSC_Certs = 
    new System.Security.Cryptography.X509Certificates.X509CertificateCollection();

Microsoft.Web.Services2.Security.X509.X509CertificateStore WS2_store =
    Microsoft.Web.Services2.Security.X509.X509CertificateStore.CurrentUserStore(
    Microsoft.Web.Services2.Security.X509.X509CertificateStore.MyStore);

WS2_store.OpenRead();

Microsoft.Web.Services2.Security.X509.X509CertificateCollection WS2_store_Certs = WS2_store.Certificates;

And then it just iterates over the WS2_store_Certs CertificateCollection and checks them all that way. A bit further on, it sets the certificates like this:

HttpWebRequest httpWebRequest = (HttpWebRequest)WebRequest.Create(url_string);
httpWebRequest.ClientCertificates = SSC_Certs;

This all looks fairly logical, even if I have no idea how it finds the certificates, but I still haven't been able to find the Java equivalent of this.

UPDATE

The connection I'm making is part of a larger application that depends on JDK 5, but I've managed to just use the sunmscapi jar to find the certificate I'm looking for. It errors when I try to connect using the windows keystore though, so I thought I got around the problem by getting the certificate I need from the windows store and inserting it in the default java one. Now I'm getting an EOFException followed by an SSLHandshakeException saying "Remote host closed connection during handshake". The ssl debug trace doesn't reveal an immediate problem to me, since the certificate I need is displayed in the certificate chain here.

It does the whole ClientKeyExchange thing, says it's finished and then the last messages I get from the debug log right after that are

[write] MD5 and SHA1 hashes:  len = 16
0000: 14 00 00 0C D3 E1 E7 3D   C2 37 2F 41 F9 38 26 CC  .......=.7/A.8&.
Padded plaintext before ENCRYPTION:  len = 32
0000: 14 00 00 0C D3 E1 E7 3D   C2 37 2F 41 F9 38 26 CC  .......=.7/A.8&.
0010: CB 10 05 A1 3D C3 13 1C   EC 39 ED 93 79 9E 4D B0  ....=....9..y.M.
AWT-EventQueue-1, WRITE: TLSv1 Handshake, length = 32
[Raw write]: length = 37
0000: 16 03 01 00 20 06 B1 D8   8F 9B 70 92 F4 AD 0D 91  .... .....p.....
0010: 25 9C 7D 3E 65 C1 8C A7   F7 DA 09 C0 84 FF F4 4A  %..>e..........J
0020: CE FD 4D 65 8D                                     ..Me.
AWT-EventQueue-1, received EOFException: error

and the code I'm using to set up the connection is

KeyStore jks = KeyStore.getInstance(KeyStore.getDefaultType());
jks.load(null, null);

KeyManagerFactory kmf = KeyManagerFactory.getInstance("SunX509");
jks.setCertificateEntry("alias", cert1); //X509Certificate obtained from windows keystore
kmf.init(jks, new char[0]);

SSLContext sslContext = SSLContext.getInstance("SSL");
sslContext.init(kmf.getKeyManagers(), new TrustManager[]{tm}, null);

sslsocketfactory = sslContext.getSocketFactory();
System.setProperty("https.proxyHost", proxyurl);
System.setProperty("https.proxyPort", proxyport);

Authenticator.setDefault(new MyAuthenticator("proxyID", "proxyPassword"));
URL url = new URL(null, urlStr, new sun.net.www.protocol.https.Handler());
HttpsURLConnection uc = (HttpsURLConnection) url.openConnection();
uc.setSSLSocketFactory(sslsocketfactory);
uc.setAllowUserInteraction(true);
uc.setRequestMethod("POST");
uc.connect();

(I haven't tried HttpClient yet because I have no idea how to find the certificate file and I'm also not sure if this will always be the same on every client system.)

ANOTHER UPDATE

I've found the CAs for the certificate I need in the WINDOWS-ROOT keystore (and checked with .verify() to see that they all check out), I've added them to the java keystore as well but still nothing changes. I guess they're supposed to go in the TrustStore, but I have yet to find a way to do that programatically. (Would prefer not to rely on end users to do this kind of thing, as all I can guarantee from them is that the certificate and CAs will be present due to the third party software mentioned at the start of this ridiculously long question.)

YET MORE UPDATES

Adding on the previous update, I've come to the conclusion that my problem must lie in the fact that my CAs are not in Java's cacerts file, so it gets the list of trusted CAs from the server, but doesn't recognise them and subsequently doesn't send a single certificate back causing the connection failure. So the problem remains, how do I get Java to either use it's keystore as truststore or add certificates to cacerts programmatically (without the need for file paths)? Because if those aren't possible that just leaves me with secret option C, voodoo. I'll start stabbing a Duke doll with needles, just in case.

Answer 1

Okay, so your question's title is How can I use certificate authentication with HttpsURLConnection? I have a working example for that. For it to work it has one prerequisite:

1. you have to have a key store which has your certificate file in it. (I know that your scenario not exactly permits this, but please just follow me here a bit so that we can narrow your problem down a little bit, because it's a bit too complicated to answer right off the bat.)

So, first get your hand on the actual certificate file. If you're on Windows 7 this can be done by the following steps:

1. open Internet Explorer,
2. open Tools (in Internet Explorer 9 it is the cog icon),
3. click Internet Options,
4. go to the Content tab,
5. click Certificates,
6. find the certificate at hand,
7. click on it and click Export and save it to a file (DER encoded binary X.509).

_{(After exporting it delete it from among the other certificates, making sure Java won't use it one way or another. I don't know if it can use it, but it couldn't hurt.)}

Now, you have to create a key store file and import the exported certificate into it, which can be done by the following.

> keytool -importcert -file <certificate> -keystore <keystore> -alias <alias>

_{(Obviously, keytool has to be on your path to work. It's part of the JDK.)}

It'll prompt you for a password (the key store's password; it doesn't have to do anything with the certificate), which I don't know how to set to "" right now, so set let it be password or whatever.

After this, with the following steps you can establish a secure connection to your endpoint via a proxy.

1. First, load the key store file.

   InputStream trustStream = new FileInputStream("path/to/<keystore>");
   char[] trustPassword = "<password>".toCharArray();
   

2. Initialize a KeyStore.

   KeyStore trustStore = KeyStore.getInstance(KeyStore.getDefaultType());
   trustStore.load(trustStream, trustPassword);
   

3. Initialize TrustManager objects. (I think these handle certificate resolution or something like that, however as far as I'm concerned this is magic.)

   TrustManagerFactory trustFactory =
       TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
   trustFactory.init(trustStore);
   TrustManager[] trustManagers = trustFactory.getTrustManagers();
   

4. Create a new SSLContext, load the TrustManager objects into it and set it as default. Take care, because SSLContext.getDefault() returns a non-modifiable instance of the class (or more like the default one can't be re-initialized, but whatever), that's why we have to use SSLContext.getInstance("SSL"). Also, don't forget to set this new SSLContext as the default, because without that the code goes poof.

   SSLContext sslContext = SSLContext.getInstance("SSL");
   sslContext.init(null, trustManagers, null);
   SSLContext.setDefault(sslContext);
   

5. Create your proxy and setup authentication for it. (Instead of using System.setProperty(...) use the Proxy class. Oh and don't be mislead by Type.HTTP.)

   Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress("<host>", <port>));
   

6. Setup authentication for your proxy. (I've used a free proxy which didn't require authentication so I couldn't test that part of the problem right now.)

   Authenticator.setDefault(new Authenticator() {
   
     @Override
     protected PasswordAuthentication getPasswordAuthentication() {
       return new PasswordAuthentication("<user>", "<password>".toCharArray());
     }
   });
   

7. Connect to your end point by passing the previously created proxy to the connection. (I've used one of my company's service's URL, which asks for a certificate—which certificate I imported in my own key store of course.)

   URL url = new URL("<endpoint>");
   URLConnection connection = url.openConnection(proxy);
   connection.connect();
   

If it doesn't work like this (you get errors) then try it with HttpsURLConnection.

   HttpsURLConnection httpsConnection = (HttpsURLConnection) connection;
   httpsConnection.setAllowUserInteraction(true);
   httpsConnection.setRequestMethod("POST");

Basically, the setAllowUserInteraction thingy kicks in if the server (where the resource pointed by the URL you connect to is located) asks for credentials, right? Now, I couldn't test just that per se, but as I see if you can get this baby working with a server that doesn't require authentication to access its resources, then you're good to go, because the server will ask you to authenticate yourself only after the connection is already established.

If after these you still receive some error, then please post it.