Rejecting mutual TLS gRPC connection based on RSA public key size

Question

I have a gRPC server which is using mutual TLS for encryption and authentication. So, every client that connects to this server provides an SSL certificate and I want to reject connections from clients who have a public key size less than 2048 bits. There seems to be no straightforward way to do this yet.

I was able to do it using a ServerInterceptor this way

public class SSLInterceptor implements ServerInterceptor {
    @Override
    public <ReqT, RespT> Listener<ReqT> interceptCall(ServerCall<ReqT, RespT> call, Metadata headers, ServerCallHandler<ReqT, RespT> next) {
        try {
            SSLSession sslSession = call.getAttributes().get(Grpc.TRANSPORT_ATTR_SSL_SESSION);
            RSAPublicKeyImpl pk = (RSAPublicKeyImpl) sslSession.getPeerCertificates()[0].getPublicKey();
            if (pk.getModulus().bitLength() < 2048) {
                // reject call
            }
            // proceed with the call
        } catch (SSLPeerUnverifiedException e) {
            // do something
        }
        ...
    }
}

This is a bad way to do it because

1. The validation is being done after the connection is already established.
2. The validation is only triggered when a request/call is made.
3. Every single call involves an extra overhead of validation.
4. In case of a validation failure, only the call is rejected but not the connection to the client.

In an ideal scenario

1. The validation is done during the connection establishment phase. (or some point during the creation of the channel between client and server)
2. The validation failure would prevent the connection from being created and not set it up and disconnect later.
3. A client is only validated once per session and all the calls made during that session do not incur any overhead.

How can I do this better?

Answer 1

You can customize certificate checking by providing your own javax.net.ssl.TrustManagerFactory to Netty's SslContextBuilder. You'd probably want to implement X509ExtendedTrustManager, do your check, and then delegate to a "real" implementation for the rest of the cert chain checking.

You can do something like this to get the default-configuration TrustManagerFactory:

TrustManagerFactory tmf = TrustManagerFactory.getInstance(
    TrustManagerFactory.getDefaultAlgorithm());
tmf.init((KeyStore) null);
// loop through tmf.getTrustManagers() checking for one implementing X509TrustManager

Answer 2

The following may help if gRPC allows you to get at the SSLEngine:

I made a mock-up using SSLEngineSimpleDemo (which is part of Oracle jssesamples.zip) to set custom AlgorithmConstraints where it creates the SSLEngine:

private void createSSLEngines() throws Exception {
    ...
    serverEngine = sslc.createSSLEngine();
    ...
    // Set custom AlgorithmConstraints on the SSL engine
    SSLParameters sslParams = sslc.getSupportedSSLParameters();
    sslParams.setAlgorithmConstraints( new MyAlgorithmConstraints() );
    serverEngine.setSSLParameters(sslParams);

The class MyAlgorithmConstraints looks something like:

import java.security.AlgorithmConstraints;
import java.security.AlgorithmParameters;
import java.security.CryptoPrimitive;
import java.security.Key;
import java.security.interfaces.RSAKey;
import java.util.Set;

public class MyAlgorithmConstraints implements AlgorithmConstraints {


    @Override
    public boolean permits(Set<CryptoPrimitive> primitives, String algorithm, AlgorithmParameters parameters) {
        return true;
    }

    @Override
    public boolean permits(Set<CryptoPrimitive> primitives, Key key) {
        boolean permitted = permittedRSAKey(primitives, key);
        return permitted;
    }

    @Override
    public boolean permits(Set<CryptoPrimitive> primitives, String algorithm, Key key, AlgorithmParameters parameters) {
        boolean permitted = permittedRSAKey(primitives, key);
        return permitted;
    }

    private boolean permittedRSAKey(Set<CryptoPrimitive> primitives, Key key) {
        boolean permitted = true;
        if (primitives.contains(CryptoPrimitive.KEY_AGREEMENT) && key instanceof RSAKey) {
            int length = ((RSAKey)key).getModulus().bitLength();
            if (length < 2040) {
                permitted = false;
                System.out.println("+*+*+* MyConstraints: short RSA key not allowed ["+length+"]");
            }
        }
        return permitted;
    }

}

Length 2040 chosen in case of leading zeroes on the key as per warning in How to get the size of a RSA key in Java but you could just as well reverse the condition to length > 1024 permitted=true.