bad_certificate error when using client certificate with retrofit (okhhtp) but works fine with curl

Question

I'm having some issues with mTLS using retrofit, okhttp-tls and Spring Boot. Communication works fine if server.ssl.client-auth is not set to need, so authentication of the server works fine.

The problem is, when server.ssl.client-auth is set to need (as it should be), trying to make an API call using retrofit results in

javax.net.ssl.SSLHandshakeException: Received fatal alert: bad_certificate

and on the server side:

*** Certificate chain
<Empty>
***
https-jsse-nio-8443-exec-4, fatal error: 43: null cert chain
javax.net.ssl.SSLHandshakeException: null cert chain

The client certificates are OK - I can make the same call using curl:

cat file_with_private_key.pem file_with_certificate.pem > combined.pem 
curl https:example.com/path --cert  combined.pem

The server then displays the certificate chain and returns the response.

The certificate configuration code looks roughly like this (I won't be able to post the actual code):

KeyStore keystore = KeyStore.getInstance("PKCS12");
keystore.load(keystoreInputStream, keystorePass);

X509Certificate clientCert = (X509Certificate) keystore.getCertificate(clientCertAlias);
PrivateKey clientKey  = (PrivateKey) keystore.getKey(clientCertAlias, keystorePass);
KeyPair clientKeyPair = new KeyPair(clientCert.getPublicKey(), clientKey);

HeldCertificate heldCertificate = new HeldCertificate(clientKeyPair, clientCert);

HandshakeCertificates handshakeCerts = new HandshakeCertificates.Builder()
        .addPlatformTrustedCertificates()
        .heldCertificate(heldCertificate)
        .build();



OkHttpClient client = new OkHttpClient.Builder()
    .sslSocketFactory(handshakeCerts.sslSocketFactory(), handshakeCerts.trustManager())
    .build();

the OkHttpClient client variable is later passed to retrofit. The exact same code worked fine when both server and client certificates were self-signed but now it doesn't work with CA signed certificates.

If I call heldCertificate.certificatePem() I get the exact same string as the BEGIN CERTIFICATE block in file_with_certificate.pem file. If I call heldCertificate.privateKeyPkcs1Pem() I get the exact same string as the BEGIN RSA PRIVATE KEY block in file_with_private_key.pem file.

I am using retrofit 2.4.0, okhttp-tls 3.14.6, java 8 and I won't be able to upgrade.

EDIT: 'file_with_certificate.pem' contains only one certificate (the leaf certificate signed by intermediate CA).

EDIT2: The pkcs12 file I am using as keystore contains the entire certificate chain (leaf certificate, intermediate certificate, root certificate) and the leaf certificate private key. Even though keytool lists 3 certificates with keytool -list -rfc, it seems that only the leaf certificate was actually loaded from the keystore.

I fixed the issue by importing each certificate from a separate file and giving each certificate a separate alias, and then iterating over aliases to load every certificate like this:

HandhsakeCertificates.Builder handshakeCertBuilder = new HandhsakeCertificates.Builder()
.addPlatformTrustedCertificates()
.heldCertificate(heldCertificate);

Collections.list(keyStore.aliases())
.foreach(alias -> {
X509Certificate cert = (X509Certificate) keystore.getCertificate(alias);
handshakeCertBuilder.addTrustedCertificate(cert);
})

EDIT3: The intermediate certificate needed to be added to server trust store, so that the server trusts client leaf certificate. Thanks @SteffenUlrich for questions pointing me in the right direction. I'm not sure why curl worked using the leaf certificate though - the only CA available in the "acceptable CA names" section of the server at the time was the root certificate, so the leaf certificate shouldn't have been accepted (because it was signed by the intermediate CA, not the root CA).

Answer 1

Looking at your setup I can conclude that your client keypair and your server trusted certificate is present in the same keystore file. I would advise to separate them into identity.jks containing the client identity aka keypair. And a truststore.jks containing all of the trusted certificates. It will make it more maintainable.

I will provide this example based on your current setup, so having in mind that everything is stored into a single keystore file. And I also noticed that you use HandshakeCertificates and HeldCertificate which are in my opinion not needed as the default jdk libraries already provide the needed objects to configure OkHttp with Retrofit

Can you try the snippet below and share you results:

KeyStore keystore = KeyStore.getInstance("PKCS12");
keystore.load(keystoreInputStream, keystorePass);

KeyManagerFactory keyManagerFactory = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
keyManagerFactory.init(keyStore, "your-key-password".toCharArray());
KeyManager[] keyManagers = keyManagerFactory.getKeyManagers();

TrustManagerFactory trustManagerFactory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
trustManagerFactory.init(keyStore);
TrustManager[] trustManagers = trustManagerFactory.getTrustManagers();

SSLContext sslContext = SSLContext.getInstance("TLS");
sslContext.init(keyManagers, trustManagers, null);
SSLSocketFactory sslSocketFactory = sslContext.getSocketFactory();

OkHttpClient client = new OkHttpClient.Builder()
    .sslSocketFactory(sslSocketFactory, trustManagers[0])
    .build();

Answer 2

The documentation example isn't great, it doesn't show the intermediate certificates that you likely need to send when you aren't using self signed certificates.

https://github.com/square/okhttp/blob/ef5d0c83f7bbd3a0c0534e7ca23cbc4ee7550f3b/okhttp-tls/README.md

A better example comes from the unit tests

https://github.com/square/okhttp/blob/f8fd4d08decf697013008b05ad7d2be10a648358/okhttp/src/test/java/okhttp3/internal/tls/ClientAuthTest.java#L331-L333

builder.heldCertificate(heldCertificate, intermediates);