Using a custom keystore with Jetty's WebSocketClientFactory

Question

I have a Java project that is connecting to a CometD server over a secure WebSocket transport which uses a self-signed certificate. I'm configuring the WebSocketClientFactory in the following way:

wssFactory.getSslContextFactory().setKeyStorePath("/path/to/my/custom.jks");
wssFactory.getSslContextFactory().setKeyStorePassword("mypass");

And then create my BayeuxClient like so:

BayeuxClient client = new BayeuxClient(
        "wss://myserver.com/cometd",
        WebSocketTransport.create(clientOptions, wssFactory));

Indeed, when this first gets loaded, things seem to be configured properly:

[DEBUG] 2012-08-07 12:58:05,786 : starting org.eclipse.jetty.websocket.WebSocketClientFactory@75cfa3d2
[DEBUG] 2012-08-07 12:58:05,786 : starting qtp2005556553{8<=0<=0/254,-1}
[DEBUG] 2012-08-07 12:58:05,788 : STARTED qtp2005556553{8<=7<=8/254,0}
[DEBUG] 2012-08-07 12:58:05,788 : starting org.eclipse.jetty.websocket.WebSocketClientFactory$WebSocketClientSelector@50c8c3b8
[DEBUG] 2012-08-07 12:58:05,795 : STARTED org.eclipse.jetty.websocket.WebSocketClientFactory$WebSocketClientSelector@50c8c3b8
[DEBUG] 2012-08-07 12:58:05,795 : Starting Thread[qtp2005556553-33 Selector0,5,main] on org.eclipse.jetty.io.nio.SelectorManager$1@346fe350
[DEBUG] 2012-08-07 12:58:05,797 : starting SslContextFactory@4240eca3(/Users/apetresc/Downloads/infrastructure.jks,null)
[INFO ] 2012-08-07 12:58:05,981 : Enabled Protocols [SSLv2Hello, SSLv3, TLSv1] of [SSLv2Hello, SSLv3, TLSv1]
[DEBUG] 2012-08-07 12:58:05,981 : Enabled Ciphers   [SSL_RSA_WITH_RC4_128_MD5, SSL_RSA_WITH_RC4_128_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_256_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_DES_CBC_SHA, SSL_DHE_RSA_WITH_DES_CBC_SHA, SSL_DHE_DSS_WITH_DES_CBC_SHA, SSL_RSA_EXPORT_WITH_RC4_40_MD5, SSL_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA, TLS_EMPTY_RENEGOTIATION_INFO_SCSV] of [SSL_RSA_WITH_RC4_128_MD5, SSL_RSA_WITH_RC4_128_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_256_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_DES_CBC_SHA, SSL_DHE_RSA_WITH_DES_CBC_SHA, SSL_DHE_DSS_WITH_DES_CBC_SHA, SSL_RSA_EXPORT_WITH_RC4_40_MD5, SSL_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA, TLS_EMPTY_RENEGOTIATION_INFO_SCSV, SSL_RSA_WITH_NULL_MD5, SSL_RSA_WITH_NULL_SHA, SSL_DH_anon_WITH_RC4_128_MD5, TLS_DH_anon_WITH_AES_128_CBC_SHA, TLS_DH_anon_WITH_AES_256_CBC_SHA, SSL_DH_anon_WITH_3DES_EDE_CBC_SHA, SSL_DH_anon_WITH_DES_CBC_SHA, SSL_DH_anon_EXPORT_WITH_RC4_40_MD5, SSL_DH_anon_EXPORT_WITH_DES40_CBC_SHA, TLS_KRB5_WITH_RC4_128_SHA, TLS_KRB5_WITH_RC4_128_MD5, TLS_KRB5_WITH_3DES_EDE_CBC_SHA, TLS_KRB5_WITH_3DES_EDE_CBC_MD5, TLS_KRB5_WITH_DES_CBC_SHA, TLS_KRB5_WITH_DES_CBC_MD5, TLS_KRB5_EXPORT_WITH_RC4_40_SHA, TLS_KRB5_EXPORT_WITH_RC4_40_MD5, TLS_KRB5_EXPORT_WITH_DES_CBC_40_SHA, TLS_KRB5_EXPORT_WITH_DES_CBC_40_MD5]
[DEBUG] 2012-08-07 12:58:05,981 : STARTED SslContextFactory@4240eca3(/Users/apetresc/Downloads/infrastructure.jks,/Users/apetresc/Downloads/infrastructure.jks)
[DEBUG] 2012-08-07 12:58:05,981 : STARTED org.eclipse.jetty.websocket.WebSocketClientFactory@75cfa3d2

And, indeed, actually using the BayeuxClient works. However, it would work even if I didn't specify a keystore at all -- it just treats it as an unsigned certificate. And, indeed, that seems to be what is happening, judging by the error logs I keep getting every few seconds:

[DEBUG] 2012-08-07 13:20:37,348 : State update: CONNECTED -> CONNECTED
[DEBUG] 2012-08-07 13:20:37,348 : Connecting, transport org.cometd.websocket.client.WebSocketTransport@53ddbcb1
[DEBUG] 2012-08-07 13:20:37,348 : [Session-1, SSL_NULL_WITH_NULL_NULL] SslConnection@79c190a2 SSL NOT_HANDSHAKING i/o/u=0/0/0 ishut=false oshut=false {WebSocketClientConnection p=WebSocketParserRFC6455@5266db4e state=START buffer= g=WebSocketGeneratorRFC6455@5ec80825 closed=false buffer=-1} NOT_HANDSHAKING filled=0/0 flushed=0/0
[DEBUG] 2012-08-07 13:20:37,348 : Sending messages [{id=50, connectionType=websocket, channel=/meta/connect, clientId=2u16ol79fcq7hqe1wu52pr0ws4aw}]
[DEBUG] 2012-08-07 13:20:37,348 : Registering WebSocketExchange {id=50, connectionType=websocket, channel=/meta/connect, clientId=2u16ol79fcq7hqe1wu52pr0ws4aw}
[DEBUG] 2012-08-07 13:20:37,348 : [Session-1, SSL_NULL_WITH_NULL_NULL] SslConnection@79c190a2 SSL NOT_HANDSHAKING i/o/u=0/0/0 ishut=false oshut=false {WebSocketClientConnection p=WebSocketParserRFC6455@5266db4e state=OPCODE buffer= g=WebSocketGeneratorRFC6455@5ec80825 closed=false buffer=-1} NOT_HANDSHAKING filled=0/0 flushed=0/0
[DEBUG] 2012-08-07 13:20:37,348 : Sending messages [{"id":"50","connectionType":"websocket","channel":"/meta/connect","clientId":"2u16ol79fcq7hqe1wu52pr0ws4aw"}]
[DEBUG] 2012-08-07 13:20:37,348 : [Session-1, SSL_NULL_WITH_NULL_NULL] SslConnection@79c190a2 SSL NOT_HANDSHAKING i/o/u=0/0/0 ishut=false oshut=false {WebSocketClientConnection p=WebSocketParserRFC6455@5266db4e state=OPCODE buffer= g=WebSocketGeneratorRFC6455@5ec80825 closed=false buffer=116} NOT_HANDSHAKING filled=0/0 flushed=0/0
[DEBUG] 2012-08-07 13:20:37,349 : [Session-1, SSL_NULL_WITH_NULL_NULL] SslConnection@79c190a2 SSL NOT_HANDSHAKING i/o/u=0/0/0 ishut=false oshut=false {WebSocketClientConnection p=WebSocketParserRFC6455@5266db4e state=OPCODE buffer=null g=WebSocketGeneratorRFC6455@5ec80825 closed=false buffer=116} NOT_HANDSHAKING filled=0/0 flushed=0/0
[DEBUG] 2012-08-07 13:20:37,349 : [Session-1, SSL_NULL_WITH_NULL_NULL] wrap OK NOT_HANDSHAKING consumed=116 produced=137
[DEBUG] 2012-08-07 13:20:37,349 : [Session-1, SSL_NULL_WITH_NULL_NULL] SslConnection@79c190a2 SSL NOT_HANDSHAKING i/o/u=0/0/0 ishut=false oshut=false {WebSocketClientConnection p=WebSocketParserRFC6455@5266db4e state=OPCODE buffer=null g=WebSocketGeneratorRFC6455@5ec80825 closed=false buffer=0} NOT_HANDSHAKING filled=0/0 flushed=137/0
[DEBUG] 2012-08-07 13:20:37,349 : [Session-1, SSL_NULL_WITH_NULL_NULL] SslConnection@79c190a2 SSL NOT_HANDSHAKING i/o/u=0/0/0 ishut=false oshut=false {WebSocketClientConnection p=WebSocketParserRFC6455@5266db4e state=OPCODE buffer=null g=WebSocketGeneratorRFC6455@5ec80825 closed=false buffer=0} NOT_HANDSHAKING filled=0/0 flushed=0/0
[DEBUG] 2012-08-07 13:20:37,349 : [Session-1, SSL_NULL_WITH_NULL_NULL] handle SslConnection@79c190a2 SSL NOT_HANDSHAKING i/o/u=0/0/0 ishut=false oshut=false {WebSocketClientConnection p=WebSocketParserRFC6455@5266db4e state=OPCODE buffer=null g=WebSocketGeneratorRFC6455@5ec80825 closed=false buffer=-1} progress=false

To me this implies that the CometD handshake is completing, but the SSL handshake is not. I can't figure out why this is the case, though; an analogous approach is working for my HTTPS calls. It's only the WSS that's giving me a headache.

May also be worth mentioning that I can reproduce this in various different environments, including an Android app and from within a Jetty servlet container.

Has anyone successfully used WSS with a self-signed certificate? Care to shed some light on what I'm doing wrong?

Answer 1

In the second code block you are not getting any "error". That is the normal output for an unconfigured SslContextFactory, which provides the null cipher suite and a always approving certificate verifier. See also here for further SslContextFactory configuration.

From the logs you have it working in both cases.

For a more generic answer to your question, when you use self-signed certificates, you may want to configure the SslContextFactory with a trust store, so that the certificate verification can use it to verify the self signed certificate.

Other than that, seems to me that you have setup your code correctly.

Answer 2

I don't know much about WebSocketClientFactory, but as far as I understand, it's for client-side configuration, where you're essentially trying to connect to a server configured with a self-signed certificate, a copy of which is stored in your custom.jks file on the client.

What you need for accepting a self-signed certificate on the client side isn't to set its keystore (which is used for client-certificate authentication from this side), but to set its truststore.

Try replacing setKeyStore* with setTrustStore*.