Apache HttpClient failing with Java 11 on macOS

Question

I'm trying to move my code from Java 8 to Java 11, this code...

 private static String  readMultiHttpsUrlResultAsString(List<String> mbRecordingIds, String level) throws Exception
{
    String result = "";
    class NaiveTrustStrategy implements TrustStrategy
    {
        @Override
        public boolean isTrusted(X509Certificate[] chain, String authType) throws CertificateException
        {
            return true;
        }
    };

    SSLContext sslcontext = org.apache.http.ssl.SSLContexts.custom()
            .loadTrustMaterial(new NaiveTrustStrategy())
            .build();

    CloseableHttpClient httpclient = HttpClients.custom()
            .setSSLSocketFactory(new SSLConnectionSocketFactory(sslcontext))
            .build();

    StringBuilder sb = new StringBuilder("?recording_ids=");
    for(String next:mbRecordingIds)
    {
        sb.append(next + ";");
    }
    sb.setLength(sb.length() - 1);
    try
    {
        String url = "https://acousticbrainz.org/api/v1"+level+sb;
        HttpGet httpget = new HttpGet(url);

        try (CloseableHttpResponse response = httpclient.execute(httpget);)
        {
            int statusCode = response.getStatusLine().getStatusCode();
            if(statusCode!=HttpURLConnection.HTTP_OK)
            {
                return "";
            }
            HttpEntity entity = response.getEntity();
            result = EntityUtils.toString(entity);
            EntityUtils.consume(entity);
        }
    }
    finally
    {
        httpclient.close();
    }
    return result;
}

}

is failing here using (AdoptOpenJdk) Java 11.0.6 on MacOS,

SSLContext sslcontext = org.apache.http.ssl.SSLContexts.custom()
            .loadTrustMaterial(new NaiveTrustStrategy())
            .build();

It runs without problem on Windows (also using AdoptOpenJdk Java 11.0.6). One difference is the Windows version uses a cutdown jre built from the jdk with jlink, whereas the MacOS version uses the AdoptOpenJDk jre build. The MacOS build is created using InfiniteKinds fork of AppBundler

This is the stacktrace:

java.lang.NoClassDefFoundError: Could not initialize class sun.security.ssl.SSLContextImpl$TLSContext
    at java.base/java.lang.Class.forName0(Native Method)
    at java.base/java.lang.Class.forName(Class.java:315)
    at java.base/java.security.Provider$Service.getImplClass(Provider.java:1848)
    at java.base/java.security.Provider$Service.newInstance(Provider.java:1824)
    at java.base/sun.security.jca.GetInstance.getInstance(GetInstance.java:236)
    at java.base/sun.security.jca.GetInstance.getInstance(GetInstance.java:164)
    at java.base/javax.net.ssl.SSLContext.getInstance(SSLContext.java:168)
    at org.apache.http.ssl.SSLContextBuilder.build(SSLContextBuilder.java:269)
    at com.jthink.songkong.analyse.acousticbrainz.AcousticBrainz.readMultiHttpsUrlResultAsString(AcousticBrainz.java:409)
    at com.jthink.songkong.analyse.acousticbrainz.AcousticBrainz.readLowLevelData(AcousticBrainz.java:373)

I'm using Apache Httpclient 4.5.3 and am using this lib because I am getting data from a webservice that requires the use of ssl.

Update I added Example test to my source code from the answer below, and modified my build to make this the start class when the application is run and it gives me this stacktrace (when bundle is run from command line with open using the java runtime embedded in the bundle by infinitekind appbundler)

Exception in thread "main" java.lang.ExceptionInInitializerError
    at java.base/javax.crypto.Cipher.getInstance(Unknown Source)
    at java.base/sun.security.ssl.JsseJce.getCipher(Unknown Source)
    at java.base/sun.security.ssl.SSLCipher.isTransformationAvailable(Unknown Source)
    at java.base/sun.security.ssl.SSLCipher.<init>(Unknown Source)
    at java.base/sun.security.ssl.SSLCipher.<clinit>(Unknown Source)
    at java.base/sun.security.ssl.CipherSuite.<clinit>(Unknown Source)
    at java.base/sun.security.ssl.SSLContextImpl.getApplicableSupportedCipherSuites(Unknown Source)
    at java.base/sun.security.ssl.SSLContextImpl$AbstractTLSContext.<clinit>(Unknown Source)
    at java.base/java.lang.Class.forName0(Native Method)
    at java.base/java.lang.Class.forName(Unknown Source)
    at java.base/java.security.Provider$Service.getImplClass(Unknown Source)
    at java.base/java.security.Provider$Service.newInstance(Unknown Source)
    at java.base/sun.security.jca.GetInstance.getInstance(Unknown Source)
    at java.base/sun.security.jca.GetInstance.getInstance(Unknown Source)
    at java.base/javax.net.ssl.SSLContext.getInstance(Unknown Source)
    at org.apache.http.ssl.SSLContextBuilder.build(SSLContextBuilder.java:389)
    at Example.main(Example.java:23)
Caused by: java.lang.SecurityException: Can not initialize cryptographic mechanism
    at java.base/javax.crypto.JceSecurity.<clinit>(Unknown Source)
    ... 17 more
Caused by: java.lang.SecurityException: Can't read cryptographic policy directory: unlimited
    at java.base/javax.crypto.JceSecurity.setupJurisdictionPolicies(Unknown Source)
    at java.base/javax.crypto.JceSecurity$1.run(Unknown Source)
    at java.base/javax.crypto.JceSecurity$1.run(Unknown Source)
    at java.base/java.security.AccessController.doPrivileged(Native Method)
    ... 18 more

which is difference to one I had before, but maybe this is the underlying cause or is this misleading ?

Whereas if I just run java -jar songkong6.9.jar it runs Example and prints out Loaded with no error, If I specify the full path from /Library/Java it also works in all cases (Java 11/Java 14/JDk and JRE)

Update Based on the answer below I have made some progress.

The JRE installed on MacOS contains a conf folder, when the JRE is added to my bundle (SongKong) using InfiniteKinds appbundler it does not have a conf folder. It doe have a lib/security folder containing a default.policy but this doesnt seem to be enough.

pauls-Mac-mini:Home paul$ ls -lR lib/security
total 704
-rw-r--r--  1 paul  admin    1253 22 Apr 14:56 blacklisted.certs
-rw-r--r--  1 paul  admin  103147 22 Apr 14:56 cacerts
-rw-r--r--  1 paul  admin    8979 22 Apr 16:01 default.policy
-rw-r--r--  1 paul  admin  233897 22 Apr 14:56 public_suffix_list.dat

After installing the built bundle if I manually copy the conf folder from the installed JRE to the Home folder of the java plugin

e.g

/Applications/SongKong.app/Contents/PlugIns/adoptopenjdk-11.jre/Contents/Home

location then both the Example code and my original code work without error when ran from bundle.

Furthermore what it seems to be looking for is the unlimited folder and its contents (the two files are actually the same), so if I delete a few files so I am left with

pauls-Mac-mini:Home paul$ pwd
/Applications/SongKong.app/Contents/PlugIns/adoptopenjdk-11.jre/Contents/Home
pauls-Mac-mini:Home paul$ ls -lR conf
total 0
drwxr-xr-x  3 paul  admin  96 22 Apr 15:14 security

conf/security:
total 0
drwxr-xr-x  3 paul  admin  96 22 Apr 15:22 policy

conf/security/policy:
total 0
drwxr-xr-x  4 paul  admin  128 22 Apr 15:28 unlimited

conf/security/policy/unlimited:
total 16
-rw-r--r--  1 paul  admin  146 22 Apr 15:06 default_US_export.policy
-rw-r--r--  1 paul  admin  193 22 Apr 15:06 default_local.policy

then it continues to work.

Problem (aside from why it doesnt work out of the box) is I assume I cannot copy files into this location for a hardened runtime app so I need to store these policy files somewhere else so they can be installed as part of the appbundler build. So as a test I have renamed conf folder conf.old folder and added following parameter to the bundle

<string>-Djava.security.policy=/Applications/SongKong.app/Contents/PlugIns/adoptopenjdk-11.jre/Contents/Home/conf.old/security/policy/unlimited/default_local.policy</string>

or to replace rather than append policy file

<string>-Djava.security.policy==/Applications/SongKong.app/Contents/PlugIns/adoptopenjdk-11.jre/Contents/Home/conf.old/security/policy/unlimited/default_local.policy</string>

But it doesn't work, I have tried various values but nothing works. The only thing that works is leaving it in conf subfolder and then it doesn't matter if I pass this parameter or not. (I also tried adding -Dsecurity.manager as another option but that just caused a new error about permissions from logging.)

Answer 1

Eventually with the help of Anish worked out it was a problem with missing policy files, and was only a problem with the bundle built with AppBunder.

The jdk code really seemed to expect a conf folder in the JRE, there was one in the OpenJDk but not once once was bundled into my app with AppBundler. So I downloaded the latest AppBundler src code and rebuilt it, rebuilt my appbundle and it was fixed, the conf folder was now included and application runs without error.

Answer 2

I have tested the code on Spring Tools Suite 4, AdoptedOpenJDK Java 11.0.6, macOS catalina 10.15.4 and Maven 3.8.1

I took a part of code which was causing problem.

Example.java :

package com.example;

import java.security.KeyManagementException;
import java.security.KeyStoreException;
import java.security.NoSuchAlgorithmException;
import java.security.cert.CertificateException;
import java.security.cert.X509Certificate;

import javax.net.ssl.SSLContext;

import org.apache.http.ssl.TrustStrategy;

class NaiveTrustStrategy implements TrustStrategy {

    public boolean isTrusted(X509Certificate[] chain, String authType) throws CertificateException {
        return true;
    }

}

public class Example {

    public static void main(String... args) throws KeyManagementException, NoSuchAlgorithmException, KeyStoreException {
        SSLContext sslcontext = org.apache.http.ssl.SSLContexts.custom().loadTrustMaterial(new NaiveTrustStrategy())
                .build();
        System.out.println("Loaded");
    }

}

Note : I took javax.net.ssl.SSLContext.

pom.xml :

<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.example</groupId>
    <artifactId>Test</artifactId>
    <version>0.0.1-SNAPSHOT</version>

    <dependencies>
        <dependency>
            <groupId>org.apache.httpcomponents</groupId>
            <artifactId>httpclient</artifactId>
            <version>4.5.3</version>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.8.1</version>
                <configuration>
                    <!-- <target>11</target> <source>11</source> -->
                    <verbose>true</verbose>
                    <fork>true</fork>
                    <executable>
                        /Library/Java/JavaVirtualMachines/adoptopenjdk-11.jdk/Contents/Home/bin/javac
                    </executable>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

The output (for Testing): No error

I have looked at this link : https://github.com/TheInfiniteKind/appbundler/

Please change the JDK/JRE to point to 11.

If problem still exists, then download the latest appbundler and try to run again.

---

From Oracle Documentation (https://www.oracle.com/java/technologies/javase-jce-all-downloads.html) :

Current versions of the JDK do not require these policy files.

JDK 9 and later ship with, and use by default, the unlimited policy files.

The unlimited policy files are required by JDK 8, 7 and 6 only.

You can download these policy files from the documentation link above.

I think the policy files can be missing or some other issue. Please verify.