Connecting to ElasticSearch Cloud 5.x with Java Client and X-Pack/HTTPS

Question

Is it possible to connect to ElasticSearch via HTTPS using X-Pack? With the switch to use certificate, the own way of connecting is no longer work. I have no problem with using certificate, but I need to know where to get one or upload the key to the cloud instance, but I cannot find information anywhere. I'm also getting no answer from anyone in the forum or IRC.

Has anyone successfully done this? There is no longer a warning when launching a 5.x instance so I assumed this is possible, but I just can't figure out how to do it. I was also told that I can no longer launch a 2.4.1 instance and only 2.4.2 is available (too unstable so far) so I'm kinda stuck not being able to launch a usable instance.

---

Update

I was told that no certificate is required since public CA is being used. However, I'm still not able to figure out how to connect to 5.1.1 instance.

Settings settings = Settings.builder()
            .put("transport.ping_schedule", "5s")
            .put("cluster.name", "<cluster_id>")
            .put("xpack.security.transport.ssl.enabled", "true")
            .put("xpack.security.user", "elastic:<password_from_cluster_creation>")
            .build();

    String hostname = "<cluster_id>.us-east-1.aws.found.io";
    TransportClient client = new PreBuiltXPackTransportClient(settings)
        .addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName(hostname), 9343));

What am I doing wrong here? I'm getting an error saying node is not available.

Exception in thread "main" NoNodeAvailableException[None of the configured nodes are available: [{#transport#-1}{xlhZqKBCQniYrN4OWzByFQ}{<cluster_id>.us-east-1.aws.found.io}{<id_address>:9343}]]

I also tried creating a user with transport_client role, but still the same error.

---

Update 2

I tried the sample code from here and it still doesn't work.

https://github.com/elastic/found-shield-example/blob/master/src/main/java/org/elasticsearch/cloud/transport/example/TransportExample.java

Here's an instance you can test with. I will destroy it later after I have figured out the issue.

---

Update 3

I figured out the issue and destroyed the test cluster. Find the solution below.

Answer 1

After some discussion with people on ES forum, I got it working finally. Here are some tips when you run into an issue with connection:

1) You probably didn't add this in the header since it's not mentioned in the documentation

.put("request.headers.X-Found-Cluster", clusterName)

2) If that still doesn't work, use the sample code from here to verify that you hostname is correct and you have the correct credential

https://github.com/elastic/found-shield-example#running

3) If you are trying to run the sample code within Eclipse, replace ${cluster.name} before you run it or it won't work.

Hope this help save people some time trying to figure out connection issue. Connecting to ES 5.x is about as easy as 2.x. The problem is that the documentation is not clear and there is no link to sample client.