Disabling SSL verification for Elastic search Restclient not working in Java

Question

I am trying to connect to elastic search hosted in a gcp box. To connect to this, there is a SSL check and I need a certificate.

However from these discussions, it is possible that we can turn off this verification.

How to disable SSL verification for Elasticsearch RestClient v6.7.0 in Java

https://discuss.elastic.co/t/host-name-does-not-match-the-certificate/186618

I had tried to remove the verification. But it throwing the following error

javax.net.ssl.SSLHandshakeException: General SSLEngine problem
    at org.elasticsearch.client.RestClient$SyncResponseListener.get(RestClient.java:947)
    at org.elasticsearch.client.RestClient.performRequest(RestClient.java:229)
    at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1762)
    at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1732)
    at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1694)
    at org.elasticsearch.client.RestHighLevelClient.search(RestHighLevelClient.java:1090)
    at org.dexter.lab.elasticUtils.ESUtils.getLastIndexedTimeStamp(ESUtils.java:44)
    at org.dexter.lab.druidUtils.DruidDelayChecker.main(DruidDelayChecker.java:357)
Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
    at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1521)
    at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:528)
    at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1197)
    at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1165)
    at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469)
    at org.apache.http.nio.reactor.ssl.SSLIOSession.doWrap(SSLIOSession.java:265)
    at org.apache.http.nio.reactor.ssl.SSLIOSession.doHandshake(SSLIOSession.java:305)
    at org.apache.http.nio.reactor.ssl.SSLIOSession.isAppInputReady(SSLIOSession.java:509)
    at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:120)
    at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)
    at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)
    at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
    at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)
    at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
    at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:588)
    at java.lang.Thread.run(Thread.java:748)
Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
    at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
    at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1709)
    at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:318)
    at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:310)
    at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1639)
    at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:223)
    at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1037)
    at sun.security.ssl.Handshaker$1.run(Handshaker.java:970)
    at sun.security.ssl.Handshaker$1.run(Handshaker.java:967)
    at java.security.AccessController.doPrivileged(Native Method)
    at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1459)
    at org.apache.http.nio.reactor.ssl.SSLIOSession.doRunTask(SSLIOSession.java:283)
    at org.apache.http.nio.reactor.ssl.SSLIOSession.doHandshake(SSLIOSession.java:353)
    ... 9 more
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:397)
    at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:302)
    at sun.security.validator.Validator.validate(Validator.java:262)
    at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324)
    at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:281)
    at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:136)
    at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1626)
    ... 17 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
    at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
    at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280)
    at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:392)
    ... 23 more

This is what I did:

I did a override to verify function and returned true. So that it won't check. This is the code for that

restClientBuilder.setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
                @Override
                public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpClientBuilder) {
                    return httpClientBuilder.setSSLHostnameVerifier(new HostnameVerifier() {
                        public boolean verify(String s, SSLSession sslSession) {
                            return true;
                        }
                    });
                }
            });

I tried with another way as well as mentioned in the elastic thread

restClientBuilder.setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
                @Override
                public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpClientBuilder) {
                    return httpClientBuilder.setSSLHostnameVerifier(NoopHostnameVerifier.INSTANCE);
                }
            });

I had generated a self signed certificate and it worked. I had enabled debug logs to see why a certificate is needed.

These are the logs after adding certificate:

***
Found trusted certificate:
[
[
  Version: V3

This was followed by details about certificate.

The problem is I cannot generate self signed certificate for production environment and there I am getting exceptions for not using certificate even after overriding.

Is there any possible work around for this ? Any help is greatly appreciated.

Thanks in advance

Answer 1

In your example you only disabled the hostname verification. The server (ElasticSearch or something on top of ElasticSearch) is sending you the public key/certificate and your Restclient tries to validate that during the ssl handshake. What you need to do is tell to your RestClient that it is OK to receive a certificate from anyone but when it receives one it should not really validate it. So you need a custom trustmanager which has the task to validate the certificate but actually it doesn't validate at all. What you need is an UnsafeX509ExtendedTrustManager, see below for the code snippets and the usage with the RestClient:

Option 1

public final class UnsafeX509ExtendedTrustManager extends X509ExtendedTrustManager {

    private static final X509ExtendedTrustManager INSTANCE = new UnsafeX509ExtendedTrustManager();
    private static final X509Certificate[] EMPTY_CERTIFICATES = new X509Certificate[0];

    private UnsafeX509ExtendedTrustManager() {}

    public static X509ExtendedTrustManager getInstance() {
        return INSTANCE;
    }

    @Override
    public void checkClientTrusted(X509Certificate[] certificates, String authType) { 

    }

    @Override
    public void checkClientTrusted(X509Certificate[] certificates, String authType, Socket socket) {
    
    }

    @Override
    public void checkClientTrusted(X509Certificate[] certificates, String authType, SSLEngine sslEngine) {

    }

    @Override
    public void checkServerTrusted(X509Certificate[] certificates, String authType) {

    }

    @Override
    public void checkServerTrusted(X509Certificate[] certificates, String authType, Socket socket) {

    }

    @Override
    public void checkServerTrusted(X509Certificate[] certificates, String authType, SSLEngine sslEngine) {

    }

    @Override
    public X509Certificate[] getAcceptedIssuers() {
        return EMPTY_CERTIFICATES;
    }

}

The above trustmanager can be supplied to the RestHighLevelClient with the following snippet:

SSLContext sslContext = SSLContext.getInstance("TLS");
sslContext.init(null, new TrustManager[]{ UnsafeX509ExtendedTrustManager.INSTANCE }, null);

RestClient restClient = RestClient
        .builder(new HttpHost("localhost", 9200, "https"))
        .setHttpClientConfigCallback(httpClientBuilder -> 
                httpClientBuilder.setSSLContext(sslContext)
                                 .setSSLHostnameVerifier((host, session) -> true))
        .build();

By the way, I don't recommend you or anyone else to use UnsafeX509ExtendedTrustManager. It is unsafe and should not be used at all in production.

Option 2

If you don't want to add the custom code to your code base but just only want to easily disable the ssl verification, you might want to give the following snippet a try. It is a library to easily generate the SSLContext or other ssl materials and it has the option to disable the ssl verification.

<dependency>
    <groupId>io.github.hakky54</groupId>
    <artifactId>sslcontext-kickstart</artifactId>
    <version>7.1.0</version>
</dependency>

Usage

SSLFactory sslFactory = SSLFactory.builder()
          .withUnsafeTrustMaterial()
          .withUnsafeHostnameVerifier()
          .build();

RestClient restClient = RestClient
        .builder(new HttpHost("localhost", 9200, "https"))
        .setHttpClientConfigCallback(httpClientBuilder -> 
                httpClientBuilder.setSSLContext(sslFactory.getSslContext())
                                 .setSSLHostnameVerifier(sslFactory.getHostnameVerifier())
        .build();