(Java) Programmatically access SSL certificates under "System Roots" on Mac OS X

Question

I am writing a Java application that does rest Api calls with remote Https site. The remote site is signed by trusted certificates. It runs well on Windows, however, it has trouble to run on OS X due to SSL certificate issues.

I did some digging and found out that the reason is related to how I initialize KeyStore object in my code with getInstance call. It only reads certificates from "System" keychain, but not from "System Roots" keychain. Below is the code snippet to print out all certificates from the key store.

// In windows use "WINDOWS-ROOT"
KeyStore osTrustManager = KeyStore.getInstance("KeychainStore");
osTrustManager.load(null, null);

Enumeration<String> enumerator = osTrustManager.aliases();
while (enumerator.hasMoreElements()) {
    String alias = enumerator.nextElement();
    if (osTrustManager.isCertificateEntry(alias)) {
        m_logger.info(String.format("%s (certificate)\n", alias));
    }
}

How can the code be changed to achieve that? Appreciate if anyone can chime in.

This is a sample of certificates under "System Roots" Screenshot from OS X

Answer 1

I do not know if there is some kind of KeyStore that allows you to access the Mac OS X System Roots certificates, but you can try another way.

In Mac OS X you can obtain a list of certificates from any keychain with the security command.

For instance, this command will give you information about the different certificates installed in the System Roots keychain:

security find-certificate -a "/System/Library/Keychains/SystemRootCertificates.keychain"

This utility has two flags, -p, which will output each certificate as PEM encoded, and -a, which allows us to filter the results by name - which can be convenient due to the great number of CA installed in the system.

The idea is use this utility from Java.

Not a long time ago, I came across a library called clienteafirma designed to deal with digital signatures.

This library has a class called AppleScript. This class basically is a wrapper around Process that allows us to run arbitrary commands.

The following code uses that class and the security command to obtain all the certificates issued by, for instance, VeriSign:

public static void main(String... args) {
  // Keychains that we can use
  final String KEYCHAIN_PATH = "/Library/Keychains/System.keychain";
  final String SYSTEM_KEYCHAIN_PATH = "/System/Library/Keychains/SystemRootCertificates.keychain";
  // Show me only certificates from VeriSign
  final String AUTHORITY = "VeriSign";
  final String OSX_SEC_COMMAND = "security find-certificate -a -p -c %AUTHORITY% %KEYCHAIN%";
  final String cmd = OSX_SEC_COMMAND.replace("%AUTHORITY%", AUTHORITY).replace("%KEYCHAIN%", SYSTEM_KEYCHAIN_PATH);
  System.out.println(cmd);
  System.out.println();

  final AppleScript script = new AppleScript(cmd);
  InputStream certificateStream = null;
  try {
    // Run script
    final String result = script.run();
    certificateStream = new ByteArrayInputStream(result.getBytes());
    // Process the output of the command
    final CertificateFactory certificateFactory = CertificateFactory.getInstance("X.509");
    final Collection<X509Certificate> certificates = (Collection<X509Certificate>) certificateFactory.generateCertificates(certificateStream);
    // Use the certificates as you need
    for (X509Certificate certificate : certificates) {
      String alias = certificate.getSubjectX500Principal().getName();
      System.out.println("Certificate: " + alias);
    }
  } catch (Throwable t) {
    t.printStackTrace();
  } finally {
    if (certificateStream != null) {
      try {
        certificateStream.close();
      } catch (IOException io) {
        io.printStackTrace();
      }
    }
  }
}

Answer 2

Requirements

• retrieve macOS anchor root certificates
• get a Java X509Certificate instance for each of it

Possible Solution

As far as I know, there is no pure Java approach to this. However, you can create a native C library that retrieves the certificates via operating system specific calls and returns them to Java via JNI.

Since macOS 10.3 there is a function SecTrustCopyAnchorCertificates in the Security framework that

retrieves the anchor (root) certificates stored by macOS.

see https://developer.apple.com/documentation/security/1401507-sectrustcopyanchorcertificates?language=objc

To construct a Java X509Certificate instance you need the certificate data in a DER-encoded format, see https://docs.oracle.com/javase/8/docs/api/java/security/cert/CertificateFactory.html#generateCertificate-java.io.InputStream-

On macOS side you get the DER encoded certificate data via SecCertificateCopyData.

Note: Since both functions SecTrustCopyAnchorCertificates and SecCertificateCopyData contain the word 'Copy', you must call CFRelease after use to avoid creating memory leaks.

The data of each certificate can be stored in a Java byte array and returned to the Java caller side.

On the Java side, you can get a CertificateFactory by calling CertificateFactory.getInstance("X.509"). Then you can convert the bytes to an X509Certificate by calling certFactory.generateCertificate(in), where in is a ByteArrayInputStream where the certificate bytes actually come from the native C lib.

Here is a self-contained example:

Native C Library

#include <stdio.h>
#include <string.h>
#include <CoreFoundation/CoreFoundation.h>
#include <Security/Security.h>
#include "com_software7_test_MacOSX509Certificates.h"


JNIEXPORT jobjectArray JNICALL Java_com_software7_test_MacOSX509Certificates_retrieveCertificates
  (JNIEnv *env, jobject obj) {
      CFArrayRef certs = NULL;
      OSStatus status = SecTrustCopyAnchorCertificates(&certs);
      if (status != noErr) {
        jclass rte = (*env)->FindClass(env, "java/lang/RuntimeException");
        if (rte != NULL)
            (*env)->ThrowNew(env, rte, "error retrieving anchor certificates"); 
        (*env)->DeleteLocalRef(env, rte);
      }
  
      CFIndex ncerts = CFArrayGetCount(certs);
      jclass byteArrayClass = (*env)->FindClass(env, "[B");
      jobjectArray array = (*env)->NewObjectArray(env, ncerts, byteArrayClass, (*env)->NewByteArray(env, 0));

      for (int i = 0; i < ncerts; i++) {   
          SecCertificateRef certRef = (SecCertificateRef)CFArrayGetValueAtIndex(certs, i);
          CFDataRef certData = SecCertificateCopyData(certRef);
          int numBytes = CFDataGetLength(certData);
          jbyteArray jCert = (*env)->NewByteArray(env, numBytes);
          (*env)->SetByteArrayRegion(env, jCert, 0, numBytes, (const jbyte *)CFDataGetBytePtr(certData));         
          CFRelease(certData);
      
          (*env)->SetObjectArrayElement(env, array, i, jCert);
          (*env)->DeleteLocalRef(env, jCert);
      }
      CFRelease(certs);  
      return array;
  }
  

Java

package com.software7.test;

import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.security.cert.CertificateException;
import java.security.cert.CertificateFactory;
import java.security.cert.X509Certificate;
import java.util.Arrays;
import java.util.List;
import java.util.stream.Collectors;

public class MacOSX509Certificates {

    static {
        System.loadLibrary("maccerts");
    }

    private native byte[][] retrieveCertificates();

    public static void main(String[] args) {
        MacOSX509Certificates mc = new MacOSX509Certificates();
        mc.retrieveAndPrint();
    }

    private void retrieveAndPrint() {
        List<X509Certificate> x509Certificates = retrieve();
        for (X509Certificate x509c : x509Certificates) {
            System.out.println(x509c.getSubjectX500Principal());
        }
    }

    private List<X509Certificate> retrieve() {
        byte[][] certs = retrieveCertificates();
        return Arrays.stream(certs)
                .<X509Certificate>map(MacOSX509Certificates::convertToX509Certificate)
                .collect(Collectors.toList());
    }

    @SuppressWarnings("unchecked")
    private static <X509Certificate> X509Certificate convertToX509Certificate(byte[] bytes) {
        try {
            CertificateFactory certFactory = CertificateFactory.getInstance("X.509");
            try (InputStream in = new ByteArrayInputStream(bytes)) {
                return (X509Certificate) certFactory.generateCertificate(in);
            }
        } catch (CertificateException | IOException e) {
            throw new RuntimeException(e);
        }
    }
}
  

Build

A build process could consist of the following steps:

export JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk-14.0.1.jdk/Contents/Home/
javac -h . com/software7/test/MacOSX509Certificates.java
clang -c -fPIC -I$JAVA_HOME/include -I$JAVA_HOME/include/darwin com_software7_test_MacOSX509Certificates.c -o com_software7_test_MacOSX509Certificates.o
clang -dynamiclib -o libmaccerts.dylib com_software7_test_MacOSX509Certificates.o -lc -framework CoreFoundation -framework Security
mv libmaccerts.dylib ../out/production/read_mac_system_certs
rm com_software7_test_MacOSX509Certificates.o
rm com/software7/test/MacOSX509Certificates.class

Test

If you run this example on a Mac it returns:

CN=Go Daddy Root Certificate Authority - G2, O="GoDaddy.com, Inc.", L=Scottsdale, ST=Arizona, C=US
CN=SwissSign Platinum CA - G2, O=SwissSign AG, C=CH
CN=AddTrust Class 1 CA Root, OU=AddTrust TTP Network, O=AddTrust AB, C=SE
CN=Global Chambersign Root, OU=http://www.chambersign.org, O=AC Camerfirma SA CIF A82743287, C=EU
...