Programmatically Obtain KeyStore from PEM

Question

How can one programmatically obtain a KeyStore from a PEM file containing both a certificate and a private key? I am attempting to provide a client certificate to a server in an HTTPS connection. I have confirmed that the client certificate works if I use openssl and keytool to obtain a jks file, which I load dynamically. I can even get it to work by dynamically reading in a p12 (PKCS12) file.

I'm looking into using the PEMReader class from BouncyCastle, but I can't get past some errors. I'm running the Java client with the -Djavax.net.debug=all option and Apache web server with the debug LogLevel. I'm not sure what to look for though. The Apache error log indicates:

... OpenSSL: Write: SSLv3 read client certificate B OpenSSL: Exit: error in SSLv3 read client certificate B Re-negotiation handshake failed: Not accepted by client!? 

The Java client program indicates:

... main, WRITE: TLSv1 Handshake, length = 48 main, waiting for close_notify or alert: state 3 main, Exception while waiting for close java.net.SocketException: Software caused connection abort: recv failed main, handling exception: java.net.SocketException: Software caused connection abort: recv failed %% Invalidated:  [Session-3, TLS_RSA_WITH_AES_128_CBC_SHA] main, SEND TLSv1 ALERT:  fatal, description = unexpected_message ... 

The client code :

public void testClientCertPEM() throws Exception {     String requestURL = "https://mydomain/authtest";     String pemPath = "C:/Users/myusername/Desktop/client.pem";      HttpsURLConnection con;      URL url = new URL(requestURL);     con = (HttpsURLConnection) url.openConnection();     con.setSSLSocketFactory(getSocketFactoryFromPEM(pemPath));     con.setRequestMethod("GET");     con.setDoInput(true);     con.setDoOutput(false);       con.connect();      String line;      BufferedReader reader = new BufferedReader(new InputStreamReader(con.getInputStream()));      while((line = reader.readLine()) != null) {         System.out.println(line);     }             reader.close();     con.disconnect(); }  public SSLSocketFactory getSocketFactoryFromPEM(String pemPath) throws Exception {     Security.addProvider(new BouncyCastleProvider());             SSLContext context = SSLContext.getInstance("TLS");      PEMReader reader = new PEMReader(new FileReader(pemPath));     X509Certificate cert = (X509Certificate) reader.readObject();              KeyStore keystore = KeyStore.getInstance("JKS");     keystore.load(null);     keystore.setCertificateEntry("alias", cert);      KeyManagerFactory kmf = KeyManagerFactory.getInstance("SunX509");     kmf.init(keystore, null);      KeyManager[] km = kmf.getKeyManagers();       context.init(km, null, null);      return context.getSocketFactory(); }  

I noticed the server is outputing SSLv3 in the log while the client is TLSv1. If I add the system property -Dhttps.protocols=SSLv3 then the client will use SSLv3 as well, but I get the same error message. I've also tried adding -Dsun.security.ssl.allowUnsafeRenegotiation=true with no change in outcome.

I've googled around and the usual answer for this question is to just use openssl and keytool first. In my case I need to read the PEM directly on the fly. I'm actually porting a C++ program that already does this, and frankly, I'm very surprised how difficult it is to do this in Java. The C++ code:

  curlpp::Easy request;   ...   request.setOpt(new Options::Url(myurl));   request.setOpt(new Options::SslVerifyPeer(false));   request.setOpt(new Options::SslCertType("PEM"));   request.setOpt(new Options::SslCert(cert));   request.perform(); 

Answer 1

I figured it out. The problem is that the X509Certificate by itself isn't sufficient. I needed to put the private key into the dynamically generated keystore as well. It doesn't seem that BouncyCastle PEMReader can handle a PEM file with both cert and private key all in one go, but it can handle each piece separately. I can read the PEM into memory myself and break it into two separate streams and then feed each one to a separate PEMReader. Since I know that the PEM files I'm dealing with will have the cert first and the private key second I can simplify the code at the cost of robustness. I also know that the END CERTIFICATE delimiter will always be surrounded with five hyphens. The implementation that works for me is:

protected static SSLSocketFactory getSocketFactoryPEM(String pemPath) throws Exception {             Security.addProvider(new BouncyCastleProvider());      SSLContext context = SSLContext.getInstance("TLS");      byte[] certAndKey = fileToBytes(new File(pemPath));      String delimiter = "-----END CERTIFICATE-----";     String[] tokens = new String(certAndKey).split(delimiter);      byte[] certBytes = tokens[0].concat(delimiter).getBytes();     byte[] keyBytes = tokens[1].getBytes();      PEMReader reader;      reader = new PEMReader(new InputStreamReader(new ByteArrayInputStream(certBytes)));     X509Certificate cert = (X509Certificate)reader.readObject();              reader = new PEMReader(new InputStreamReader(new ByteArrayInputStream(keyBytes)));     PrivateKey key = (PrivateKey)reader.readObject();              KeyStore keystore = KeyStore.getInstance("JKS");     keystore.load(null);     keystore.setCertificateEntry("cert-alias", cert);     keystore.setKeyEntry("key-alias", key, "changeit".toCharArray(), new Certificate[] {cert});      KeyManagerFactory kmf = KeyManagerFactory.getInstance("SunX509");     kmf.init(keystore, "changeit".toCharArray());      KeyManager[] km = kmf.getKeyManagers();       context.init(km, null, null);      return context.getSocketFactory(); } 

Update: It seems this can be done without BouncyCastle:

    byte[] certAndKey = fileToBytes(new File(pemPath));     byte[] certBytes = parseDERFromPEM(certAndKey, "-----BEGIN CERTIFICATE-----", "-----END CERTIFICATE-----");     byte[] keyBytes = parseDERFromPEM(certAndKey, "-----BEGIN PRIVATE KEY-----", "-----END PRIVATE KEY-----");      X509Certificate cert = generateCertificateFromDER(certBytes);                   RSAPrivateKey key  = generatePrivateKeyFromDER(keyBytes); 

...

protected static byte[] parseDERFromPEM(byte[] pem, String beginDelimiter, String endDelimiter) {     String data = new String(pem);     String[] tokens = data.split(beginDelimiter);     tokens = tokens[1].split(endDelimiter);     return DatatypeConverter.parseBase64Binary(tokens[0]);         }  protected static RSAPrivateKey generatePrivateKeyFromDER(byte[] keyBytes) throws InvalidKeySpecException, NoSuchAlgorithmException {     PKCS8EncodedKeySpec spec = new PKCS8EncodedKeySpec(keyBytes);      KeyFactory factory = KeyFactory.getInstance("RSA");      return (RSAPrivateKey)factory.generatePrivate(spec);         }  protected static X509Certificate generateCertificateFromDER(byte[] certBytes) throws CertificateException {     CertificateFactory factory = CertificateFactory.getInstance("X.509");      return (X509Certificate)factory.generateCertificate(new ByteArrayInputStream(certBytes));       } 

Answer 2

Although the answer of Ryan works well I want to provide an alternative for other developers as I faced a similar challenge in the past where I also needed to handle encrypted private keys in pem format. I have created a library to simplify loading pem files and creating SSLSocketFactory or SSLContext out of it, see here: GitHub - SSLContext Kickstart I hope you like it :)

The pem files can be loaded with the following snippet:

var keyManager = PemUtils.loadIdentityMaterial("certificate-chain.pem", "private-key.pem"); var trustManager = PemUtils.loadTrustMaterial("some-trusted-certificate.pem");  var sslFactory = SSLFactory.builder()           .withIdentityMaterial(keyManager)           .withTrustMaterial(trustManager)           .build();  var sslContext = sslFactory.getSslContext(); var sslSocketFactory = sslFactory.getSslSocketFactory(); 

Coming back to your main question, with the above snippet it is not needed to create a keystore object from the pem files. It will take care of that under the covers and it will map it to a KeyManager instance.