Java8, HttpClient, receiving "Received fatal alert: handshake_failure"

Question

We're trying to access the following URL (just an example) https://broadly.vice.com/en_us/article/eating-out-as-a-feminist but we're receiving "Received fatal alert: handshake_failure".

We're using JDK 1.8.60, HttpClient 4.3.6 and already replaced the jce-policy to the unlimited policy.

After activating -Djavax.net.debug=all we receive:

Allow unsafe renegotiation: false
Allow legacy hello messages: true
Is initial handshake: true
Is secure renegotiation: false
Ignoring unsupported cipher suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 for TLSv1
Ignoring unsupported cipher suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 for TLSv1
Ignoring unsupported cipher suite: TLS_RSA_WITH_AES_256_CBC_SHA256 for TLSv1
Ignoring unsupported cipher suite: TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384 for TLSv1
Ignoring unsupported cipher suite: TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384 for TLSv1
Ignoring unsupported cipher suite: TLS_DHE_RSA_WITH_AES_256_CBC_SHA256 for TLSv1
Ignoring unsupported cipher suite: TLS_DHE_DSS_WITH_AES_256_CBC_SHA256 for TLSv1
Ignoring unsupported cipher suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 for TLSv1.1
Ignoring unsupported cipher suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 for TLSv1.1
Ignoring unsupported cipher suite: TLS_RSA_WITH_AES_256_CBC_SHA256 for TLSv1.1
Ignoring unsupported cipher suite: TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384 for TLSv1.1
Ignoring unsupported cipher suite: TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384 for TLSv1.1
Ignoring unsupported cipher suite: TLS_DHE_RSA_WITH_AES_256_CBC_SHA256 for TLSv1.1
Ignoring unsupported cipher suite: TLS_DHE_DSS_WITH_AES_256_CBC_SHA256 for TLSv1.1
%% No cached client session
*** ClientHello, TLSv1.2
RandomCookie:  GMT: 1443469228 bytes = { 243, 104, 4, 214, 135, 213, 2, 212, 116, 191, 239, 20, 101, 244, 129, 121, 19, 205, 49, 37, 253, 149, 203, 37, 52, 13, 87, 59 }
Session ID:  {}
Cipher Suites: [TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, TLS_RSA_WITH_AES_256_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384, TLS_DHE_RSA_WITH_AES_256_CBC_SHA256, TLS_DHE_DSS_WITH_AES_256_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDH_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_DSS_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_DSS_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, TLS_EMPTY_RENEGOTIATION_INFO_SCSV]
Compression Methods:  { 0 }
Extension elliptic_curves, curve names: {secp256r1, sect163k1, sect163r2, secp192r1, secp224r1, sect233k1, sect233r1, sect283k1, sect283r1, secp384r1, sect409k1, sect409r1, secp521r1, sect571k1, sect571r1, secp160k1, secp160r1, secp160r2, sect163r1, secp192k1, sect193r1, sect193r2, secp224k1, sect239k1, secp256k1}
Extension ec_point_formats, formats: [uncompressed]
Extension signature_algorithms, signature_algorithms: SHA512withECDSA, SHA512withRSA, SHA384withECDSA, SHA384withRSA, SHA256withECDSA, SHA256withRSA, SHA224withECDSA, SHA224withRSA, SHA1withECDSA, SHA1withRSA, SHA1withDSA, MD5withRSA
***
[write] MD5 and SHA1 hashes:  len = 237
0000: 01 00 00 E9 03 03 56 0A   98 AC F3 68 04 D6 87 D5  ......V....h....
0010: 02 D4 74 BF EF 14 65 F4   81 79 13 CD 31 25 FD 95  ..t...e..y..1%..
0020: CB 25 34 0D 57 3B 00 00   64 C0 24 C0 28 00 3D C0  .%4.W;..d.$.(.=.
0030: 26 C0 2A 00 6B 00 6A C0   0A C0 14 00 35 C0 05 C0  &.*.k.j.....5...

0040: 0F 00 39 00 38 C0 23 C0   27 00 3C C0 25 C0 29 00  ..9.8.#.'.<.%.).
0050: 67 00 40 C0 09 C0 13 00   2F C0 04 C0 0E 00 33 00  g.@...../.....3.
0060: 32 C0 2C C0 2B C0 30 00   9D C0 2E C0 32 00 9F 00  2.,.+.0.....2...
0070: A3 C0 2F 00 9C C0 2D C0   31 00 9E 00 A2 C0 08 C0  ../...-.1.......
0080: 12 00 0A C0 03 C0 0D 00   16 00 13 00 FF 01 00 00  ................
0090: 5C 00 0A 00 34 00 32 00   17 00 01 00 03 00 13 00  \...4.2.........
00A0: 15 00 06 00 07 00 09 00   0A 00 18 00 0B 00 0C 00  ................
00B0: 19 00 0D 00 0E 00 0F 00   10 00 11 00 02 00 12 00  ................
00C0: 04 00 05 00 14 00 08 00   16 00 0B 00 02 01 00 00  ................
00D0: 0D 00 1A 00 18 06 03 06   01 05 03 05 01 04 03 04  ................
00E0: 01 03 03 03 01 02 03 02   01 02 02 01 01           .............
hz.cached.thread-1, WRITE: TLSv1.2 Handshake, length = 237
[Raw write]: length = 242
0000: 16 03 03 00 ED 01 00 00   E9 03 03 56 0A 98 AC F3  ...........V....
0010: 68 04 D6 87 D5 02 D4 74   BF EF 14 65 F4 81 79 13  h......t...e..y.
0020: CD 31 25 FD 95 CB 25 34   0D 57 3B 00 00 64 C0 24  .1%...%4.W;..d.$
0030: C0 28 00 3D C0 26 C0 2A   00 6B 00 6A C0 0A C0 14  .(.=.&.*.k.j....
0040: 00 35 C0 05 C0 0F 00 39   00 38 C0 23 C0 27 00 3C  .5.....9.8.#.'.<
0050: C0 25 C0 29 00 67 00 40   C0 09 C0 13 00 2F C0 04  .%.).g.@...../..
0060: C0 0E 00 33 00 32 C0 2C   C0 2B C0 30 00 9D C0 2E  ...3.2.,.+.0....
0070: C0 32 00 9F 00 A3 C0 2F   00 9C C0 2D C0 31 00 9E  .2...../...-.1..
0080: 00 A2 C0 08 C0 12 00 0A   C0 03 C0 0D 00 16 00 13  ................
0090: 00 FF 01 00 00 5C 00 0A   00 34 00 32 00 17 00 01  .....\...4.2....
00A0: 00 03 00 13 00 15 00 06   00 07 00 09 00 0A 00 18  ................
00B0: 00 0B 00 0C 00 19 00 0D   00 0E 00 0F 00 10 00 11  ................
00C0: 00 02 00 12 00 04 00 05   00 14 00 08 00 16 00 0B  ................
00D0: 00 02 01 00 00 0D 00 1A   00 18 06 03 06 01 05 03  ................
00E0: 05 01 04 03 04 01 03 03   03 01 02 03 02 01 02 02  ................
00F0: 01 01                                            

[Raw read]: length = 5
0000: 15 03 03 00 02                                     .....
[Raw read]: length = 2
0000: 02 28                                              .(
hz.cached.thread-1, READ: TLSv1.2 Alert, length = 2
hz.cached.thread-1, RECV TLSv1.2 ALERT:  fatal, handshake_failure
hz.cached.thread-1, called closeSocket() 

hz.cached.thread-1, handling exception: javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure
hz.cached.thread-1, called close()
hz.cached.thread-1, called closeInternal(true)

ERROR net.HttpClientCommonsImpl - Failed to access page https://broadly.vice.com/en_us/article/eating-out-as-a-feminist
javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure
    at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) ~[?:1.8.0_60]
    at sun.security.ssl.Alerts.getSSLException(Alerts.java:154) ~[?:1.8.0_60]
    at sun.security.ssl.SSLSocketImpl.recvAlert(SSLSocketImpl.java:2023) ~[?:1.8.0_60]
    at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1125) ~[?:1.8.0_60]
    at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1375) ~[?:1.8.0_60]
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403) ~[?:1.8.0_60]
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387) ~[?:1.8.0_60]
    at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:535) ~[httpclient-4.3.6.jar:4.3.6]
    at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:403) ~[httpclient-4.3.6.jar:4.3.6]
    at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:472) ~[httpclient-4.3.6.jar:4.3.6]
    at org.apache.http.conn.scheme.SchemeSocketFactoryAdaptor.connectSocket(SchemeSocketFactoryAdaptor.java:65) ~[httpclient-4.3.6.jar:4.3.6]
    at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177) ~[httpclient-4.3.6.jar:4.3.6]
    at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304) ~[httpclient-4.3.6.jar:4.3.6]
    at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611) ~[httpclient-4.3.6.jar:4.3.6]
    at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446) ~[httpclient-4.3.6.jar:4.3.6]
    at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:863) ~[httpclient-4.3.6.jar:4.3.6]
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82) ~[httpclient-4.3.6.jar:4.3.6]

How to overcome the handshake_failure error?

Answer 1

For those arriving late here, like me:

The problem happens because of the issue Server Name Indication (SNI) Support

But this issue was already fixed, so what's the problem? The problem happens because you are using the deprecated SSLSocketFactory instead of the newer SSLConnectionSocketFactory.

The issue was fixed inSSLConnectionSocketFactory but not in the deprecated SSLSocketFactory.

So, if you can, use SSLConnectionSocketFactory and you should be fine.

SSLSocketFactory and SSLConnectionSocketFactory are obviously not interchangeable, and if you are like me, you may have a lot of code to change before you can replace one with the other, so what do we do?

What I did: I extended SSLSocketFactory, and overrided a method in a way to match the patch that fixed the issue:

public class PatchedSSLSocketFactory extends SSLSocketFactory {
    public PatchedSSLSocketFactory(
        final SSLContext sslContext,
        final X509HostnameVerifier hostnameVerifier) {
        super(sslContext, hostnameVerifier);
    }

    @Override
    public Socket createSocket(final HttpContext context) throws IOException {
        return SocketFactory.getDefault().createSocket();
    }
}  

Answer 2

According to ssllabs, the server is all right, and it works java7+. Notably, clients that don't support SNI will fail talking to the server. In your debug dump, SNI is indeed missing, and that's likely the problem.

SNI should be enabled by default on java7+. Your stacktrace shows that indeed it's on java8. This link might help you.

Answer 3

We experience the same issue with httpclient and java 8 trying to access an amazon server with https.

Seems to be related to this bug :

https://bugs.openjdk.java.net/browse/JDK-8072464

This is an openJDK bug but we also experience it with Oracle 8u60