How to use MSCAPI provider for client SSL authentication

Question

I am writing a program which needs to make HTTPS connections to a web server, where SSL Client authentication needs to be used.

The users of this program will use certificates from the windows environment to authenticate themselves.

I've found plenty of examples showing how to set up client authentication, and it works fine if I first export my certificate to pkcs12 format, but I don't wish to force my users to do that. However when I try to use the MSCAPI it always bombs out with an exception:

javax.net.ssl.SSLHandshakeException: Error signing certificate verify
        at sun.security.ssl.Alerts.getSSLException(Unknown Source)
        at sun.security.ssl.SSLSocketImpl.fatal(Unknown Source)
        at sun.security.ssl.Handshaker.fatalSE(Unknown Source)
        at sun.security.ssl.ClientHandshaker.serverHelloDone(Unknown Source)
        at sun.security.ssl.ClientHandshaker.processMessage(Unknown Source)
        at sun.security.ssl.Handshaker.processLoop(Unknown Source)
        at sun.security.ssl.Handshaker.process_record(Unknown Source)
        at sun.security.ssl.SSLSocketImpl.readRecord(Unknown Source)
        at sun.security.ssl.SSLSocketImpl.readDataRecord(Unknown Source)
        at sun.security.ssl.AppInputStream.read(Unknown Source)
        at java.io.BufferedInputStream.fill(Unknown Source)
        at java.io.BufferedInputStream.read1(Unknown Source)
        at java.io.BufferedInputStream.read(Unknown Source)
        at sun.net.www.http.HttpClient.parseHTTPHeader(Unknown Source)
        at sun.net.www.http.HttpClient.parseHTTP(Unknown Source)
        at sun.net.www.http.HttpClient.parseHTTP(Unknown Source)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
        at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(Unknown Source)
        at com.example.Win2.main(Win2.java:62)
Caused by: java.security.SignatureException: Bad Key.

        at sun.security.mscapi.RSASignature.signHash(Native Method)
        at sun.security.mscapi.RSASignature.engineSign(RSASignature.java:390)
        at java.security.Signature$Delegate.engineSign(Unknown Source)
        at java.security.Signature.sign(Unknown Source)
        at sun.security.ssl.RSASignature.engineSign(Unknown Source)
        at java.security.Signature$Delegate.engineSign(Unknown Source)
        at java.security.Signature.sign(Unknown Source)
        at sun.security.ssl.HandshakeMessage$CertificateVerify.<init>(Unknown Source)
        ... 16 more

I can't really tell what might be wrong with the key from that exception.

I have made a tiny test program to reproduce the problem I am having:

String passwd = .....";
URL url = new URL("https://.........");

KeyManagerFactory keyManagerFactory = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
KeyStore keyStore = KeyStore.getInstance("Windows-MY");
keyStore.load(null, passwd.toCharArray());
keyManagerFactory.init(keyStore, passwd.toCharArray());

SSLContext context = SSLContext.getInstance("TLS");
context.init(keyManagerFactory.getKeyManagers(), null, null);
SSLSocketFactory socketFactory = context.getSocketFactory();

HttpsURLConnection conn = (HttpsURLConnection) url.openConnection();
conn.setSSLSocketFactory(socketFactory);

There is clearly something about the APIs I don't understand here. How does it know which of the keys in the keystore I want to use? I was expecting for Windows to prompt me, as it does for other applications which need to authenticate, but I suspect instead it is just choosing the first one it finds.

Do I need to implement my own key manager so it can choose which key will be used?

If I iterate through the keystore, I can see the keys in it and can extract one by calling getKey. To complicate matters, there are multiple keys in the store with the same alias (but differing validity). Other (non-java) applications, eg. Chrome, seem to be able to determine which keys to use somehow.

EDIT: I forgot to mention, I am using Java 1.7.

Answer 1

The responsible for choosing which key in the keystore would be used is the KeyManager object. You can get a vector of those objects calling keyManagerFactory.getKeyManagers().

The libraries usually get the first key entry they find in the store (could be compatible with the server certificate presented in this case). The MS-CAPI API does no different.

To select which key in the keystore you want to use, you should do 3 things:

1. implement the interface X509KeyManager

2. make the method chooseClientAlias of the above interface return the desired alias of your key

3. set the object to your SSLContext.

Remind that your keystore must contain all the chain of certificates starting at your personal certificate up to a root authority. You should use the certmgr.msc program to import the certificates and/or check if all of them are present in the folders Personal (your certificate), Intermediate Certification Authorities (any middle CA in your chain) and Trusted Root Certification Authorities.

It's also important to set your trust keystore - it stores the root certificates you trust, this store would be used to verify the server certificate. In case of MS-CAPI you would use the KeyStore.getInstance("Windows-ROOT") command to get it (the certs in the Trusted Root Certification Authorities folder).

Modifying your code to accomplish that:

URL url = new URL("https://.........");

KeyManagerFactory keyManagerFactory = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
KeyStore keyStore = KeyStore.getInstance("Windows-MY");
keyStore.load(null, null);
keyManagerFactory.init(keyStore);

/* You must also set your trust store */
KeyStore ts = KeyStore.getInstance("Windows-ROOT");
TrustManagerFactory tmf = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
tmf.init(ts);

/* Here you can implement a way to set your key alias 
** You can run through all key entries and implement a way
** to prompt the user to choose one - for simplicity I just set a
** name*/
String alias = "user1_alias";

/* Get your current KeyManager from the factory */
final X509KeyManager okm = (X509KeyManager)keyManagerFactory.getKeyManagers()[0];

/* Implement the Interface X509KeyManager */
X509KeyManager km = new X509KeyManager() {
    public String chooseClientAlias(String[] keyType, Principal[] issuers, Socket socket) {
         /* Implement your own logic to choose the alias
            according to the validity if the case, 
            or use the entry id or any other way, you can get 
            those values outside this class*/
         return alias;
    }

    public X509Certificate[] getCertificateChain(String alias) {
         return okm.getCertificateChain(alias);
    }
   /* Implement the other methods of the interface using the okm object */
};
SSLContext context = SSLContext.getInstance("TLS");
/* set the keymanager in the SSLContext */
context.init(new KeyManager[]{km}, tmf.getTrustManagers(), new SecureRandom());
SSLSocketFactory socketFactory = context.getSocketFactory();

HttpsURLConnection conn = (HttpsURLConnection) url.openConnection();
conn.setSSLSocketFactory(socketFactory);