SSL Handshake Failed - Java 1.8

Question

Just letting folks know about an issue I had that many seemed to have had after upgrading to Java 1.8. Not all of the solutions are the same hence posting how I resolved this.

But first... This is not a solution worthy of production systems since security is being effectively downgraded. However, if you are blocked testing etc. it is probably quite suitable.

My issue was that no matter what I did... enabled SSLv3 etc. I always received

"javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure".

Here are the steps I took to 'solve' this.

First, I discovered which cipher the server was using. I did this via openssl.

openssl s_client -host yourproblemhost.com -port 443

This yields (at the end...)

SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : RC4-MD5

Now.. what do we use 'Java-wise' to enable that cipher?

Oracle link

In that link, it has the names and their Java counterpart. So for RC4-MD5, we have SSL_RSA_WITH_RC4_128_MD5.

ok good. Now I added a System property.

-Dhttps.cipherSuites=SSL_RSA_WITH_RC4_128_MD5

And in my code...

Security.setProperty("jdk.tls.disabledAlgorithms", "" /*disabledAlgorithms */ );

Again.. this is an absolute last resort 'fix'... But if you're hitting your head aganst a wall to get it running (for testing), I hope it comes in useful.

Answer 1

With JDK 1.8.0_51 release RC4 is no longer supported from Java as client (also as server) to negotiate SSL handshake, RC4 is considered weak (and compromised ) cipher and that is the reason for removal

http://bugs.java.com/view_bug.do?bug_id=8076221

You can still however enable it by removing RC4 from jdk.tls.disabledAlgorithms from your Java security config or progamatically enabling them using setEnabledCipherSuites() method

However better solution would be to update the server configuration (if it is under your control) to upgrade to stronger Ciphers

RC4 is now considered as a compromised cipher. RC4 cipher suites have been removed from both client and server default enabled cipher suite list in Oracle JSSE implementation. These cipher suites can still be enabled by SSLEngine.setEnabledCipherSuites() and SSLSocket.setEnabledCipherSuites() methods.

As to your approach on setting it by using Security.setProperty(), it is not reliable way because the fields which hold disabled algorithms are static and final, So if that class gets loaded first you don't have controll over it, you could alternatively try by creating a properties file

like this

## override it to remove RC4, in disabledcipher.properties
jdk.tls.disabledAlgorithms=DHE

and in your JVM, you could refer it as system property like this

java -Djava.security.properties=disabledcipher.properties blah...

Answer 2

RC4 was effectively cracked - 14 years ago.

The Fluhrer, Mantin and Shamir (FMS) attack, published in their 2001 paper "Weaknesses in the Key Scheduling Algorithm of RC4", takes advantage of a weakness in the RC4 key scheduling algorithm to reconstruct the key from encrypted messages.

The problem isn't in Java 8.

The problem is your server is using RC4.