Elasticsearch in Docker container cluster

Question

I want to run 2 instances of Elasticsearch on 2 different hosts.

I have built my own Docker image based on Ubuntu 14.04 and the 1.3.2 version of Elasticsearch. If I run 2 ES containers on 1 host, each instance can see and communicate with the other; but when I run 2 instances of ES on 2 different hosts, it didn't work. The 9300 port of the container is bind to the 9300 host's port.

Is it possible to create an ES cluster with my configuration?

Answer 1

I was able to get clustering working using unicast across two docker hosts. I just happen to be using the ehazlett/elasticsearch image, but I do not think this should matter all that much. The really important bit seems to be setting the network.publish_host setting to a public or routable IP its docker host.

Configuration

---

docker-host-01

eth0: 192.168.1.10 Docker version 1.4.1, build 5bc2ff8/1.4.1 

docker-host-02

eth0: 192.168.1.20 Docker version 1.4.1, build 5bc2ff8/1.4.1 

Building the Cluster

---

On Docker Host 01

docker run -d \   -p 9200:9200 \   -p 9300:9300 \   ehazlett/elasticsearch \   --cluster.name=unicast \   --network.publish_host=192.168.1.10 \   --discovery.zen.ping.multicast.enabled=false \   --discovery.zen.ping.unicast.hosts=192.168.1.20 \   --discovery.zen.ping.timeout=3s \   --discovery.zen.minimum_master_nodes=1 

On Docker Host 02

docker run -d \   -p 9200:9200 \   -p 9300:9300 \   ehazlett/elasticsearch \   --cluster.name=unicast \   --network.publish_host=192.168.1.20 \   --discovery.zen.ping.multicast.enabled=false \   --discovery.zen.ping.unicast.hosts=192.168.1.10 \   --discovery.zen.ping.timeout=3s \   --discovery.zen.minimum_master_nodes=1 

Answer 2

Using docker-compose is much easier than running it manually in command line:

elasticsearch_master:     image: elasticsearch:latest     command: "elasticsearch -Des.cluster.name=workagram -Des.node.master=true -Des.node.data=false"     environment:        - ES_HEAP_SIZE=512m     ports:       - "9200:9200"       - "9300:9300"  elasticsearch1:     image: elasticsearch:latest     command: "elasticsearch -Des.cluster.name=workagram -Des.discovery.zen.ping.unicast.hosts=elasticsearch_master"     links:       - elasticsearch_master     volumes:       - "/opt/elasticsearch/data"     environment:        - ES_HEAP_SIZE=512m elasticsearch2:     image: elasticsearch:latest     command: "elasticsearch -Des.cluster.name=workagram -Des.discovery.zen.ping.unicast.hosts=elasticsearch_master"     links:       - elasticsearch_master     volumes:       - "/opt/elasticsearch/data"     environment:        - ES_HEAP_SIZE=512m