What is the cause of "java.security.UnrecoverableKeyException: Rejected by the jceks.key.serialFilter or jdk.serialFilter property"?

Question

When I call the function under java.security.KeyStore:

public final Key getKey(String alias, char[] password)

I got the following error:

java.security.UnrecoverableKeyException: Rejected by the jceks.key.serialFilter or jdk.serialFilter property
  at com.sun.crypto.provider.KeyProtector.unseal(KeyProtector.java:352)
  at com.sun.crypto.provider.JceKeyStore.engineGetKey(JceKeyStore.java:136)
  at java.security.KeyStore.getKey(KeyStore.java:1023)

This error does not exist in any Java document, and only happens intermittenly. What is the cause of this error and how to fix it?

UPDATE: now it is revealed by @zeal that it is related to http://www.oracle.com/technetwork/java/javase/8u171-relnotes-4308888.html#JDK-8189997. So without additional configuration only a few options of Key implementation can be used. However in the release note I found a statement:

Customers storing a SecretKey that does not serialize to the above types must modify the filter to make the key extractable.

This seems to be something new as it indicates that the key's serialization can be overridden by the program, is it the only way to make other key types compatible with JCEKS keystore?

Answer 1

its causing because of issue in latest java version JDK-8 build 171. there has been a switch from jks to pkcs12 and it's the open issue at java side (latest JDK-8 build 171) in jcrypto: https://github.com/jcryptool/core/issues/120.

workaround for this is to switch JRE 8 build 171 build to JRE 8 build 144/121

Answer 2

you can open /jre/lib/security/java.security file and try to find property jceks.key.serialFilter and add your filter class/package there.

At my end Old entry for jceks.key.serialFilter property was:

jceks.key.serialFilter = java.lang.Enum;java.security.KeyRep;java.security.KeyRep$Type;javax.crypto.spec.SecretKeySpec;!*

Added org.apache.hadoop.crypto.key.**, so New Entry for jceks.key.serialFilter property is:

jceks.key.serialFilter = java.lang.Enum;java.security.KeyRep;java.security.KeyRep$Type;javax.crypto.spec.SecretKeySpec;org.apache.hadoop.crypto.key.**;!*