Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

how to run multiple instances of elasticsearch on one host

I have several machines each with 128 GB of ram, each host is running a single instance of Elasticsearch. I would like to run another data node on each host and allocate around 30 GB to the jvm heap.

I know I have to create a separate config file .yml and data directory..etc. My question is do I need to modify the service wrapper so that each node will be started/ stopped seperatly?

I am running ES version 1.3 on Centos 6.5

thank you

like image 580
Yasir Avatar asked Feb 12 '15 16:02

Yasir


People also ask

How do I run multiple instances of Elasticsearch?

You can create multiple config dirs if you wish and then start elasticsearch with -Epath. conf=config1 . For each instance you need to create 'config, data, and logs' folders. e.g. create config1, config2, ...

Can I run multiple Elasticsearch nodes on the same machine?

In this case is better to have a larger node instance. But to run multiple nodes in the same hosts you need to have a different elasticsearch. yml for every node with separated data and log folders, there isn't a way to use the same elasticsearch. yml to run multiple nodes at the same time.

How many nodes should I have Elasticsearch?

three node is best as if you have one fail node you will still have your cluster running. if you have one node in cluster then also it is fine, but when it goes down your cluster is down.


1 Answers

You need to prepare two elasticsearch.yml config files to configure settings accordingly and specify these files when startup up the two nodes.

bin/elasticsearch -Des.config=$ES_HOME/config/elasticsearch.1.yml
bin/elasticsearch -Des.config=$ES_HOME/config/elasticsearch.2.yml

At least the following should be set differently for the two nodes:

http.port
transport.tcp.port
path_data
path_logs
path_pid
node.name

The following needs to point to the other in both files to allow the nodes to find each other:

discovery.zen.ping.unicast.hosts: '127.0.0.1:9302'

EDIT: the property is now deprecated, look at : https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-discovery-settings.html

See this blog and this discussion

like image 57
centic Avatar answered Oct 23 '22 00:10

centic