Spring boot 1.4.x and custom CharsetProvider

Question

I'm trying to register a custom CharsetProvider to be able to use X-Gsm7Bit encoding. I use https://github.com/OpenSmpp/opensmpp/tree/master/charset/src/main/java/org/smpp/charset provider from Logica. To register this new charset provider I use META-INF/services/java.nio.charsets.spi.CharsetProvider file with the content org.smpp.charset.Gsm7BitCharsetProvider.

I can't make it working. The sources of a test application are here https://github.com/asmsoft/provider

I get java.util.ServiceConfigurationError: java.nio.charset.spi.CharsetProvider: Provider org.smpp.charset.Gsm7BitCharsetProvider not found when I start it as a fat jar

mvn clean
mvn package
java -jar target/provider-1.0-SNAPSHOT.jar

If I start it with mvn spring-boot:run I get java.io.UnsupportedEncodingException: X-Gsm7Bit

And everything works well when I start the application with my IDE.

Currently I solved my problem as follows: I've put jar providing custom charset into JAVA_HOME/jre/lib/ext and everything works as expected again, the charset is being registered on the boot.

I'm not happy with this solution and I'd like to ask for your help.

Answer 1

I think you've encountered a bug in the JDK. The javadoc for CharsetProvider says:

Charset providers are looked up via the current thread's context class loader

However, the code that looks up the providers tells a different story:

ClassLoader cl = ClassLoader.getSystemClassLoader();
ServiceLoader<CharsetProvider> sl =
    ServiceLoader.load(CharsetProvider.class, cl);

As you can see, it's using the system class loader and not the thread's context class loader.

It works in your IDE because your application's classes and their dependencies are all available to the system class loader. When you package your application in a fat jar, it fails as your application's classes and their dependencies are then available to Spring Boot's class loader which is a child of the system class loader. You'd have the same problem in any environment where another class loader is used, for example a traditional war deployment to a servlet container.

To avoid the problem, you need to make the provider available to the system class loader. As you've discovered, putting its jar in JAVA_HOME/jre/lib/ext is one way to achieve that. Other options include using a shaded jar rather than a fat jar, or post-processing the fat jar to add the provider's classes directly to the root of the jar rather than as a nested jar.

Answer 2

As Andy Wilkinson mentioned, you probably hit a bug, namely this one (created in 2002): https://bugs.openjdk.java.net/browse/JDK-4619777

I don't know if this solves your problem, but you can get the Charset by using the CharsetProvider directly, something in the line of:

Charset charset = new CharsetProvider().charsetForName("...");

You might even create a utility method to get a Charset with manual fallbacks to CharsetProviders:

Charset getCharset(String charsetName) {
    if (Charset.isSupported(charsetName)) {
        return Charset.forName(charsetName);
    }

    Charset charset = new org.smpp.charset.Gsm7BitCharsetProvider().charsetForName(charsetName);
    if (charset != null) { return charset; }
    charset = new com.beetstra.jutf7.CharsetProvider().charsetForName(charsetName);
    if (charset != null) { return charset; }
    // more fallbacks here ...
    return null;
}

Although this would only be a workaround for the mentioned JDK bug, since this is exactly the logic that would automatically be performed by registering CharsetProviders.