Robolectric test that uses OkHttp for real HTTP requests throws java.lang.NullPointerException: No password supplied for PKCS#12 KeyStore

Question

I'm using Robolectric 4.3.1 (testImplementation "org.robolectric:robolectric:4.3.1") to create an Android sqlite environment for my integration tests. My system uses OkHttp (implementation 'com.squareup.okhttp3:okhttp:3.14.7') for real HTTP requests. I'm not using MockWebServer.

After I upgraded to the Android 10 SDK, I had to update my unit test JVM to JDK 9, per the Robolectric instructions:

Running tests on Android API 29 now strictly requires a Java9 runtime or newer. If you are seeing errors about unsupported Java version when running tests on API 29 via Android Studio; you can use the 'JRE' field in the Run Configuration dialog to configure a newer Java runtime. See https://developer.android.com/studio/run/rundebugconfig for more background.

However, now my integration tests fail with:

java.lang.NullPointerException: No password supplied for PKCS#12 KeyStore.

    at org.bouncycastle.jcajce.provider.keystore.pkcs12.PKCS12KeyStoreSpi.engineLoad(Unknown Source)
    at java.base/java.security.KeyStore.load(KeyStore.java:1479)
    at java.base/sun.security.ssl.TrustStoreManager$TrustAnchorManager.loadKeyStore(TrustStoreManager.java:367)
    at java.base/sun.security.ssl.TrustStoreManager$TrustAnchorManager.getTrustedCerts(TrustStoreManager.java:315)
    at java.base/sun.security.ssl.TrustStoreManager.getTrustedCerts(TrustStoreManager.java:59)
    at java.base/sun.security.ssl.TrustManagerFactoryImpl.engineInit(TrustManagerFactoryImpl.java:51)
    at java.base/javax.net.ssl.TrustManagerFactory.init(TrustManagerFactory.java:278)
    at okhttp3.internal.Util.platformTrustManager(Util.java:640)
    at okhttp3.OkHttpClient.<init>(OkHttpClient.java:228)
    at okhttp3.OkHttpClient.<init>(OkHttpClient.java:202)

How can I make real HTTP calls again?

Answer 1

TLDR

to workaround this problem, set the javax.net.ssl.trustStoreType system property to JKS:

-Djavax.net.ssl.trustStoreType=JKS

Details

I found this workaround:

After trying many things I finally found one workaround:

System.setProperty("javax.net.ssl.trustStore", "NONE")

MockWebServer()

The tests are passing with this additional configuration.

However, when I tried that workaround, all of my HTTP calls failed with the following ConnectException instead:

java.net.ConnectException: Failed to connect to myhost.com:443
    at okhttp3.internal.connection.RealConnection.connectSocket(RealConnection.java:265)
    at okhttp3.internal.connection.RealConnection.connect(RealConnection.java:183)
    at okhttp3.internal.connection.ExchangeFinder.findConnection(ExchangeFinder.java:224)
    at okhttp3.internal.connection.ExchangeFinder.findHealthyConnection(ExchangeFinder.java:108)
    at okhttp3.internal.connection.ExchangeFinder.find(ExchangeFinder.java:88)
    at okhttp3.internal.connection.Transmitter.newExchange(Transmitter.java:169)
    at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:41)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
    at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:94)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
    at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
    at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:88)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
    at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
    at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:229)
    at okhttp3.RealCall.execute(RealCall.java:81)

I suspect that OkHttp rightly denies all TLS connections because it doesn't trust them since the workaround suggests setting the javax.net.ssl.trustStore to NONE.

I also tried specifying the default password for java truststores, changeit via this workaround:

Workaround: -Djavax.net.ssl.trustStorePassword=changeit

But then I got the following exception when trying to instantiate an OkHttpClient:

java.lang.AssertionError: No System TLS

    at okhttp3.internal.Util.platformTrustManager(Util.java:648)
    at okhttp3.OkHttpClient.<init>(OkHttpClient.java:228)
    at okhttp3.OkHttpClient.<init>(OkHttpClient.java:202)

which was caused by:

Caused by: java.security.KeyStoreException: problem accessing trust store
    at java.base/sun.security.ssl.TrustManagerFactoryImpl.engineInit(TrustManagerFactoryImpl.java:75)
    at java.base/javax.net.ssl.TrustManagerFactory.init(TrustManagerFactory.java:278)
    at okhttp3.internal.Util.platformTrustManager(Util.java:640)
    ... 22 more
Caused by: java.io.IOException: stream does not represent a PKCS12 key store
    at org.bouncycastle.jcajce.provider.keystore.pkcs12.PKCS12KeyStoreSpi.engineLoad(Unknown Source)
    at java.base/java.security.KeyStore.load(KeyStore.java:1479)
    at java.base/sun.security.ssl.TrustStoreManager$TrustAnchorManager.loadKeyStore(TrustStoreManager.java:367)
    at java.base/sun.security.ssl.TrustStoreManager$TrustAnchorManager.getTrustedCerts(TrustStoreManager.java:315)
    at java.base/sun.security.ssl.TrustStoreManager.getTrustedCerts(TrustStoreManager.java:59)
    at java.base/sun.security.ssl.TrustManagerFactoryImpl.engineInit(TrustManagerFactoryImpl.java:51)
    ... 24 more

This was a great clue. I started debugging the JDK 9 source with Android Studio, and I noticed that when I ran with org.junit.runners.BlockJUnit4ClassRunner, my KeyStore.keystoreSpi was a sun.security.pkcs12.PKCS12KeyStore$DualFormatPKCS12, but when I ran with org.robolectric.RobolectricTestRunner my KeyStore.keystoreSpi was a org.bouncycastle.jcajce.provider.keystore.pkcs12.PKCS12KeyStoreSpi$BCPKCS12KeyStore.

According to JEP-229:

This feature changes the default keystore type from JKS to PKCS12. By default, new keystores will be created in the PKCS12 keystore format. Existing keystores will not change and keystore applications can continue to explicitly specify the keystore type they require.

Existing applications must not be disrupted. Keystores tend to be long-lived, so we need to support access across several JDK releases. Applications that access keystores created by earlier JDK releases must run unaltered on JDK 9. Similarly, applications that access keystores created by JDK 9 should run unaltered on earlier JDK releases.

This requirement is achieved by introducing a keystore detection mechanism that understands both the JKS and PKCS12 formats. A keystore's format is examined before it is loaded to determine its type and then the appropriate keystore implementation is used to access it. The mechanism is enabled by default but can be disabled if required.

Support for this keystore-detection mechanism may be backported to earlier JDK releases.

Thus, the classic $JAVA_HOME/lib/security/cacerts is still a Java Key Store, which I could verify:

$ file /Library/Java/JavaVirtualMachines/jdk-9.0.4.jdk/Contents/Home/lib/security/cacerts
/Library/Java/JavaVirtualMachines/jdk-9.0.4.jdk/Contents/Home/lib/security/cacerts: Java KeyStore

However, since the JDK wants to default to PKCS12 keystores, the JDK's DualFormatPKCS12 will fall back to a JKS if reading a file as PKCS12 fails. Bouncycastle assumes that when the javax.net.ssl.trustStoreType is pkcs12 that is really what we mean.

Thus, to workaround this problem, set the javax.net.ssl.trustStoreType system property to JKS:

-Djavax.net.ssl.trustStoreType=JKS

I filed issues with bouncycastle and robolectric about this.