Receiving SSLHandshakeException: handshake_failure despite my client ignoring all certs

Question

I have a Java program that connects to a webserver using SSL/TLS, and sends various HTTP requests over that connection. The server is localhost and is using a self-signed cert, but my code is using custom TrustManagers, and ignores invalid certificates. It has worked perfectly until now.

The only difference on the server is that it used to run jboss 6 and is now running jboss 7. I'm not sure if this is a configuration issue, or whether there is a problem with my code, but I get the same errors if I try to connect using other Java-based programs like WebScarab or ZAP.

In any case, is there anything I can do to my code to get around this problem? Here is the error in full:

Received fatal alert: handshake_failure
javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure
        at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Unknown Source)
        at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Unknown Source)
        at com.sun.net.ssl.internal.ssl.SSLSocketImpl.recvAlert(Unknown Source)
        at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(Unknown Source)
        at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(Unknown Source)
        at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(Unknown Source)
        at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(Unknown Source)
        at sun.net.www.protocol.https.HttpsClient.afterConnect(Unknown Source)
        at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(Unknown Source)
        at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(Unknown Source)

Here are the debug messages before the failure:

main, WRITE: TLSv1 Handshake, length = 75
main, WRITE: SSLv2 client hello message, length = 101
main, READ: TLSv1 Alert, length = 2
main, RECV TLSv1 ALERT: fatal, handshake_failure

Answer 1

So I found the problem. There might be a bug in Java, but the client seems to initiate a TLSv1 Handshake, but then sends an SSLv2 client hello message, at which point the server rejects the connection.

This happens even if you create your SSLContext with an instance of TLS:

SSLContext sslContext = SSLContext.getInstance("TLS");

The solution is to set a system property before any connection attempts are made:

System.setProperty("https.protocols", "TLSv1");

There are probably other solutions to it, but this one worked for me.

Answer 2

The info you provide is very little as well as your stack trace.
I'll take a guess here.
What I suspect is that in the new server the protocol is TLSv1 while your clients try to connect with SSLv3 (or less) and as a result the handshake fails.

Change you clients to use higher version of TLS or
Make your webserver support SSLv3 as well. I know how to do this in Tomcat but not in JBoss.

If this doesn't work update the post with more info (and a full stack trace).
You should enable ssl debug info -Djavax.net.debug=ssl