No cipher suites in common while establishing a secure connection

Question

I'm trying to establish a secure connection between two Java projects but I'm getting a SSLHandshakeException (no cipher suites in common). This are the methods to create sockets in both sides:

Client:

private SSLSocket getSocketConnection() throws SSLConnectionException {
    try {

        /* Load properties */
        String keystore = properties.getProperty("controller.keystore");
        String passphrase = properties.getProperty("controller.passphrase");
        String host = properties.getProperty("controller.host");
        int port = Integer.parseInt(properties
                .getProperty("controller.port"));

        /* Create keystore */
        KeyStore keyStore = KeyStore.getInstance(KeyStore.getDefaultType());
        keyStore.load(new FileInputStream(keystore), passphrase.toCharArray());

        /* Get factory for the given keystore */
        TrustManagerFactory tmf = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
        tmf.init(keyStore);
        SSLContext ctx = SSLContext.getInstance("SSL");
        ctx.init(null, tmf.getTrustManagers(), null);
        SSLSocketFactory factory = ctx.getSocketFactory();

        return (SSLSocket) factory.createSocket(host, port);
    } catch (Exception e) {
        throw new SSLConnectionException(
                "Problem connecting with remote controller: "
                        + e.getMessage(), e.getCause());
    }
}

Server:

private SSLServerSocket getServerSocket() throws SSLConnectionException {
    try {

        /* Load properties */
        Properties properties = getProperties("controller.properties");

        String keystore = properties.getProperty("controller.keystore");
        String passphrase = properties.getProperty("controller.passphrase");
        int port = Integer.parseInt(properties
                .getProperty("controller.port"));

        /* Create keystore */
        KeyStore keyStore = KeyStore.getInstance(KeyStore.getDefaultType());
        keyStore.load(new FileInputStream(keystore), passphrase.toCharArray());

        /* Get factory for the given keystore */
        TrustManagerFactory tmf = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
        tmf.init(keyStore);
        SSLContext ctx = SSLContext.getInstance("SSL");
        ctx.init(null, tmf.getTrustManagers(), null);
        SSLServerSocketFactory factory = ctx.getServerSocketFactory();

        return (SSLServerSocket) factory.createServerSocket(port);
    } catch (Exception e) {
        throw new SSLConnectionException(
                "Problem starting auth server: "
                        + e.getMessage(), e.getCause());
    }
}

I have a RSA key generated with keytool. This code load it from disk.

What I'm doing wrong?

UPDATE: I added the a call to setEnabledCipherSuites in both sides with this array:

String enableThese[] =
{
    "SSL_RSA_WITH_3DES_EDE_CBC_SHA",
    "SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA",
    "SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA"
};

I get the same result.

Answer 1

On the server side, you're not initialising the keystore/keymanagers, only the truststore/trustmanagers: ctx.init(null, tmf.getTrustManagers(), null).

On the server, initialising the keymanager is always necessary to configure the server certificate. Initialising the truststore is only necessary when you want to use client-certificate authentication. (There are more details in this question for the difference between keymanager and trustmanager.)

Without any keymanager configured, there is no RSA or DSA based certificate available, so no cipher suite that rely on a certificate for authentication (all the ones enabled by default are) are available. Hence, you get no cipher suites in common between the client and the server.

You'd need something like this:

KeyManagerFactory kmf = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
kmf.init(keystore, password.toCharArray()); // That's the key's password, if different.
// ...
ctx.init(kmf.getKeyManagers(), null, null);

It's not clear from your example, but you shouldn't of course use the same keystore on the client (as a truststore) and on the server side (as a keystore): the private key should only be known to the server, and doesn't need to be in the client's trust store.

---

EDIT: (I'll try to re-explain in a different way, since it wasn't clear for everyone. Perhaps it might help.)

The following code initialises the SSLContext with a null array of key managers (the first argument): ctx.init(null, tmf.getTrustManagers(), null). (There is no default key manager.)

The key manager is what manages your (private) keys and certificates, on the side where the code is running. On the server, the key manager is what's responsible for handling the server certificate and its private key. The key manager is itself usually initialised by the "keystore keystore". "keystore" in Java can have multiple meanings. One of the meaning of keystore is the entity into which keys and certificates can be stored, typically a file. Such a keystore can be used to initialise a trust manager¹ (in which case it's referred to as the truststore) or a key manager (in which case it's referred to as a keystore). Sorry, not my choice of names, but that's the way the system properties are called.

When the server is configured with a null key manager, it is configured without any certificate and associated private key. Therefore, it doesn't have any RSA or DSA certificate. Therefore, it won't be able to use any of the *_RSA_* or *_DSS_* cipher suites, whether they've been explicitly enabled or not (they will be disabled automatically by lack of certificate to use with them). This effectively discards any cipher suite enabled by default (or any such cipher suite enabled explicitly anyway). Hence, there is "no cipher suite in common".

In short, an SSLContext on the server side needs to be configured with a certificate and its private key². This is done by configuring its key manager. In turn, this is often done by using a keystore with a KeyManagerFactory (not a TrustManagerFactory).

_{1: The trust manager uses local trust anchors (e.g. trusted CA certificates) to evaluate trust in a remote party (i.e. a server trusting a client certificate or a client trusting a server certificate).}

_{2: Some cipher suites supported by the JSSE don't need certificates, but they're either anonymous cipher suites (insecure) or Kerberos cipher suites (which need to be set up differently altogether). Both are disabled by default.}