Java ssl/https client using a self-signed certificate

Question

I am attempting to write a Java https client using jdk version 1.6.0_32. I have a self-signed public certificate that I have imported into a new truststore. The issue is that I keep getting "Exception in thread "main" javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake" The SSL debug output is as follows:

C:\Users\csheets\eclispe_workspace\sdpweb\InstallSSLCert>java TestCert
keyStore is :
keyStore type is : jks
keyStore provider is :
init keystore
init keymanager of type SunX509
trustStore is: c:\users\csheets\4startrust.ts
trustStore type is : jks
trustStore provider is :
init truststore
adding as trusted cert:
Subject: CN=4starserver.servehttp.com
Issuer:  CN=4STAR
Algorithm: RSA; Serial number: 0x200000001
Valid from Mon May 14 11:25:15 MDT 2012 until Tue May 14 11:25:15 MDT 2013

trigger seeding of SecureRandom
done seeding SecureRandom
Allow unsafe renegotiation: true
Allow legacy hello messages: true
Is initial handshake: true
Is secure renegotiation: false
%% No cached client session
*** ClientHello, TLSv1
RandomCookie:  GMT: 1320442869 bytes = { 175, 184, 30, 195, 10, 55, 219, 232, 23
, 237, 63, 239, 83, 49, 125, 80, 10, 174, 112, 210, 61, 53, 232, 66, 179, 22, 16
1, 80 }
Session ID:  {}
Cipher Suites: [SSL_RSA_WITH_RC4_128_MD5, SSL_RSA_WITH_RC4_128_SHA, TLS_RSA_WITH
_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC
_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_
DSS_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_DES_CBC_SHA, SSL_DHE_RSA_WITH_DES_CBC_SH
A, SSL_DHE_DSS_WITH_DES_CBC_SHA, SSL_RSA_EXPORT_WITH_RC4_40_MD5, SSL_RSA_EXPORT_
WITH_DES40_CBC_SHA, SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_DSS_EXPORT_WI
TH_DES40_CBC_SHA, TLS_EMPTY_RENEGOTIATION_INFO_SCSV]
Compression Methods:  { 0 }
***
main, WRITE: TLSv1 Handshake, length = 75
main, WRITE: SSLv2 client hello message, length = 101
main, READ: SSLv3 Handshake, length = 527
*** ServerHello, SSLv3
RandomCookie:  GMT: 1320442987 bytes = { 158, 143, 79, 29, 193, 160, 122, 201, 8
1, 67, 17, 26, 159, 243, 54, 202, 255, 156, 125, 121, 132, 174, 17, 202, 222, 65
, 252, 77 }
Session ID:  {131, 30, 0, 0, 6, 235, 145, 226, 5, 214, 118, 217, 18, 123, 46, 20
4, 51, 182, 211, 225, 48, 172, 95, 70, 144, 4, 178, 150, 166, 75, 166, 29}
Cipher Suite: SSL_RSA_WITH_RC4_128_SHA
Compression Method: 0
Extension renegotiation_info, renegotiated_connection: <empty>
***
%% Created:  [Session-1, SSL_RSA_WITH_RC4_128_SHA]
** SSL_RSA_WITH_RC4_128_SHA
*** Certificate chain
chain [0] = [
[
  Version: V3
  Subject: CN=4starserver.servehttp.com
  Signature Algorithm: SHA1withRSA, OID = 1.2.840.113549.1.1.5

  Key:  Sun RSA public key, 1024 bits
  modulus: 129409122589634486230897608496688768265641316152022572826296995250983
  80968933262586507340653723460037384941316405007365056646964455523390263136350462
  59738068084572819329229707448458528878467480278641098016863640927986379246142644
  62745346179244207665720440347282685862962453661441013596685879879277368109494267

public exponent: 65537
Validity: [From: Mon May 14 11:25:15 MDT 2012,
           To: Tue May 14 11:25:15 MDT 2013]
Issuer: CN=4STAR
SerialNumber: [    02000000 01]

]
Algorithm: [SHA1withRSA]
Signature:
0000: 00 8E A1 F4 58 22 F2 C2   A9 1D C6 CB 5A 23 F5 A5  ....X"......Z#..
0010: 02 3A C9 FF 83 96 1A 13   3A 0F 59 D5 1E 1F 56 85  .:......:.Y...V.
0020: AB 4A 46 8D F3 43 E8 BA   B3 F9 B7 8C FB 76 AD D5  .JF..C.......v..
0030: 9F 15 47 DC 30 72 F9 BA   B1 FF DA 2C 25 89 FF 30  ..G.0r.....,%..0
0040: C4 4F BA D6 0C B9 30 10   B0 4B 74 EF 8A F4 5D F1  .O....0..Kt...].
0050: AC 2C 47 D9 C2 F5 A0 AF   CE 8B 76 53 36 A3 BE 11  .,G.......vS6...
0060: 7E BA 1F 4A 67 C1 69 EF   C3 E6 32 E2 0D 09 93 66  ...Jg.i...2....f
0070: 92 21 66 88 95 CA BD C8   FF CF 79 9D 7E F3 DC E0  .!f.......y.....

]
***
Found trusted certificate:
[
[
  Version: V3
  Subject: CN=4starserver.servehttp.com
  Signature Algorithm: SHA1withRSA, OID = 1.2.840.113549.1.1.5

  Key:  Sun RSA public key, 1024 bits
  modulus: 129409122589634486230897608496688768265641316152022572826296995250983
80968933262586507340653723460037384941316405007365056646964455523390263136350462
59738068084572819329229707448458528878467480278641098016863640927986379246142644
62745346179244207665720440347282685862962453661441013596685879879277368109494267

 public exponent: 65537
 Validity: [From: Mon May 14 11:25:15 MDT 2012,
           To: Tue May 14 11:25:15 MDT 2013]
 Issuer: CN=4STAR
 SerialNumber: [    02000000 01]

 ]
Algorithm: [SHA1withRSA]
Signature:
0000: 00 8E A1 F4 58 22 F2 C2   A9 1D C6 CB 5A 23 F5 A5  ....X"......Z#..
0010: 02 3A C9 FF 83 96 1A 13   3A 0F 59 D5 1E 1F 56 85  .:......:.Y...V.
0020: AB 4A 46 8D F3 43 E8 BA   B3 F9 B7 8C FB 76 AD D5  .JF..C.......v..
0030: 9F 15 47 DC 30 72 F9 BA   B1 FF DA 2C 25 89 FF 30  ..G.0r.....,%..0
0040: C4 4F BA D6 0C B9 30 10   B0 4B 74 EF 8A F4 5D F1  .O....0..Kt...].
0050: AC 2C 47 D9 C2 F5 A0 AF   CE 8B 76 53 36 A3 BE 11  .,G.......vS6...
0060: 7E BA 1F 4A 67 C1 69 EF   C3 E6 32 E2 0D 09 93 66  ...Jg.i...2....f
0070: 92 21 66 88 95 CA BD C8   FF CF 79 9D 7E F3 DC E0  .!f.......y.....

]
*** ServerHelloDone
*** ClientKeyExchange, RSA PreMasterSecret, SSLv3
main, WRITE: SSLv3 Handshake, length = 132
SESSION KEYGEN:
PreMaster Secret:
0000: 03 00 07 86 97 89 23 A4   73 85 54 59 A4 76 DD 85  ......#.s.TY.v..
0010: 12 1A 28 1B 71 CC 7A B2   EE 0F 65 60 26 30 6C B4  ..(.q.z...e`&0l.
0020: B4 92 2D 15 50 51 E5 10   77 96 8E B0 4F 30 57 73  ..-.PQ..w...O0Ws
CONNECTION KEYGEN:
Client Nonce:
0000: 4F B4 5C F5 AF B8 1E C3   0A 37 DB E8 17 ED 3F EF  O.\......7....?.
0010: 53 31 7D 50 0A AE 70 D2   3D 35 E8 42 B3 16 A1 50  S1.P..p.=5.B...P
Server Nonce:
0000: 4F B4 5C 6B 9E 8F 4F 1D   C1 A0 7A C9 51 43 11 1A  O.\k..O...z.QC..
0010: 9F F3 36 CA FF 9C 7D 79   84 AE 11 CA DE 41 FC 4D  ..6....y.....A.M
Master Secret:
0000: 60 59 16 75 E0 5E 4B 64   D6 6B 56 18 9B F2 C8 7A  `Y.u.^Kd.kV....z
0010: F8 DF 65 C6 C0 12 92 62   15 A1 4E 5F 53 D3 02 EF  ..e....b..N_S...
0020: 9B EF ED FD 1E 01 61 6F   AC 39 E0 5B AD 87 BF 25  ......ao.9.[...%
Client MAC write Secret:
0000: C1 36 79 97 E6 71 22 D0   27 D0 41 88 F9 F5 8D C2  .6y..q".'.A.....
0010: EA A3 97 FB                                        ....
Server MAC write Secret:
0000: C4 00 15 49 31 29 B2 F3   06 90 59 F0 5A 4D 3D 45  ...I1)....Y.ZM=E
0010: 32 B2 B6 83                                        2...
Client write key:
0000: E6 46 87 A2 16 52 04 11   73 15 E8 23 9F E6 02 A3  .F...R..s..#....
Server write key:
0000: 04 63 1D 64 E7 25 FC E4   53 FC 43 04 33 3C ED 6E  .c.d.%..S.C.3<.n
... no IV used for this cipher
main, WRITE: SSLv3 Change Cipher Spec, length = 1
*** Finished
verify_data:  { 61, 221, 238, 253, 97, 36, 152, 79, 254, 95, 226, 136, 55, 16, 2
07, 66, 58, 197, 233, 254, 125, 99, 11, 0, 138, 51, 139, 62, 175, 123, 52, 167,
131, 216, 245, 97 }
***
main, WRITE: SSLv3 Handshake, length = 60
main, received EOFException: error
main, handling exception: javax.net.ssl.SSLHandshakeException: Remote host close
d connection during handshake
main, SEND SSLv3 ALERT:  fatal, description = handshake_failure
main, WRITE: SSLv3 Alert, length = 22
main, called closeSocket()
Exception in thread "main" javax.net.ssl.SSLHandshakeException: Remote host clos
ed connection during handshake
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(Unknown Source)

    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(Un
known Source)
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(Unknown Sou
rce)
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(Unknown Sou
rce)
    at sun.net.www.protocol.https.HttpsClient.afterConnect(Unknown Source)
    at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect
(Unknown Source)
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown So
urce)
    at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(Unkn
own Source)
    at TestCert.main(TestCert.java:92)
Caused by: java.io.EOFException: SSL peer shut down incorrectly
    at com.sun.net.ssl.internal.ssl.InputRecord.read(Unknown Source)
    ... 9 more

Not sure how to interpret all of this information but it does seem like my certificate is being found and trusted - what does the ClientKeyExchange mean - could it be that the server is expecting client authentication or is this just the certificate info being exchanged. The whole Java keystore, trustore and self-signed certificates seems to create a lot of complexity and confusion - at least for me.

My test client code is as follows:

import java.io.*;
import java.net.URL;
import java.net.URLConnection;

import javax.net.ssl.*;
import javax.net.ssl.X509TrustManager;
import java.security.cert.X509Certificate;
import java.security.*;

public class TestCert {
public static void main(String[] args) throws Exception {
    System.setProperty("javax.net.debug", "ssl");
    System.setProperty("sun.security.ssl.allowUnsafeRenegotiation", "true");
    System.setProperty("javax.net.ssl.trustStore", "c:\\users\\csheets\\4startrust.ts");
    System.setProperty("javax.net.ssl.trustStorePassword", "mypassword");


    SSLSocketFactory sslsocketfactory = (SSLSocketFactory) SSLSocketFactory.getDefault();

    URL url = new URL("https://4starserver.servehttp.com:777/?username=0&password=0&command=WEBAUTH&TRAN=2&MERCHANT=9999999999119911&FNAME=TONY&LNAME=PISCOPO&CC=4111111111111111&EXP=0613&AMOUNT=99.98");
    HttpsURLConnection conn = (HttpsURLConnection)url.openConnection();
    conn.setSSLSocketFactory(sslsocketfactory);

    InputStream inputstream = conn.getInputStream();
    InputStreamReader inputstreamreader = new InputStreamReader(inputstream);
    BufferedReader bufferedreader = new BufferedReader(inputstreamreader);

    String string = null;
    while ((string = bufferedreader.readLine()) != null) {
        System.out.println("Received " + string);
    }
}
}

I have tried other things like creating my own TrustManager and TrustManager as well but that has not seemed to work either. Any assistance would be greatly appreciated.

UPDATE: The server port was changed to 443 so could test with Qualys SSL Lab test and I got the following results:

Common Error Messages

Connect timed out - server did not respond to our connection request
No route to host - unable to reach the server
Unable to connect to server - failed to connect to the server
Unrecognized SSL message, plaintext connection? - the server responded with plain-text HTTP on HTTPS port
Received fatal alert: handshake_failure - this is either a faulty SSL server or some other server listening on port 443; if the SSL version of the web site works in your browser, please report this issue to us
Known Issues

Could not generate DH keypair - due to a known problem with the underlying SSL library (Sun's JSSE implementation) we are unable to assess the sites that offer only DHE handshakes stronger than 1024 bits.

The strange thing is that I seem to get a response back from the server if I just put the URL https://server.com/ in the browser - but on the server evidently it seems that there are several requests / connections being made. Seems like there be something not quite right on the server side?

Answer 1

You certificate is trusted (using your custom trust store), this is not a trust manager issue. You don't need to allow unsafe renegotiation (nor to specify the default SSLSocketFactory).

There seems to be a problem with the TLS setup, since it works with System.setProperty("https.protocols", "SSLv3");. You also get a handshake problem if you force -tls1 with openssl s_client.

If you're in control of the server, I'd suggest trying to put it on port 443 and test it via the Qualys SSL labs test (which is more detailed and catches more issues than many other tests available).