Switching between the nodes in ElasticSearch using JEST client

Question

I have 3 Nodes in my ElasticSearch Cluster. eg. 10.10.0.1, 10.10.0.2, 10.10.0.3

Now I am trying to connect to the Cluster using Jest Client API and giving the IP (10.10.0.1) of one of the above nodes. Now if the Node (10.10.0.1) is down, so does that mean that I would not able to access the cluster?

How to handle this kind of problem where we have multiple nodes and we try to connect to the cluster from the code using one IP address?

Or

Should I go and create other connection giving the other IP address? This I feel is very crude way of doing it because, what if we have 100 node, in that case we would end up trying to connect to 100 nodes one after other.

Answer 1

I would take a look at creating at what is known as a client node. A client node has node.data set to false, so while it is a full member of the cluster it stores no data locally. This both off-loads query processing from nodes handling document indexing and provides the beginnings of a load balancing approach.

The data-less node has significantly lower chance of failure, given that most software failure scenarios for Elasticsearch revolve around blowing out your JVM heap and that is much less likely when you are only doing query processing and not also maintaining indexes. Further, this approach can be extended to a type of local load balancer.

If you were to run your client node elasticsearch instance on your application server you effectively provide local load balancing. The node will send out the queries to all other nodes in the cluster that need to be included in a specific query. And since the software is running on the same server as your application software you eliminate the failure mode of the node you are connecting to failing while your app server is still up.

This architectural approach is discussed in some detail here:

https://blog.liip.ch/archive/2013/07/19/on-elasticsearch-performance.html

And a bit more here on client nodes:

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-node.html

Alternatively, you could use the native Java Elasticsearch api which does not use the REST interface - this allows your Java application to connect as a member node of the cluster, in which case it will know about all the other nodes and will be able to route your queries to the correct node. I'd take a look at both the client node and the transport node approaches:

http://www.elasticsearch.org/guide/en/elasticsearch/client/java-api/current/client.html#node-client

http://www.elasticsearch.org/guide/en/elasticsearch/client/java-api/current/client.html#transport-client