Elasticsearch clustering behind UFW firewall

Question

I have an Elasticsearch cluster running on two different Digital Ocean droplets. They are both set up for private networking, an I have a Mongo DB replica set running just fine with UFW rules set up to only accept connections on the relevant ports from the droplets' specific (private) IP addresses.

However I am not able to get a green Elasticsearch cluster health using the same method, only yellow. This means that the nodes are not able to connect to each other.

In elasaticsearch.yml (on both machines) I have disabled multicast and am using unicast to connect to the droplet's internal IP addresses. When I set up the firewall to accept all connections on port 9300 (ufw allow 9300) this works just fine, and the cluster health is reported as green. However, when I restrict the rule to allow only from the actual IP addresses, just as with the Mongo DB replica set, it doesn't work. I have tried with both the public and private addresses, and with IPv4 and IPv6.

What am I missing here?

Answer 1

IPV6 is preferred by default. You can change this behavior by setting java.net.preferIPv4Stack system property to true.
Also you have to see, by default ES bind to anyLocalAddress (typically 0.0.0.0 or ::0). You can change this by setting network.bind_host with the correct ip address.

Reference [1.3] » Modules » Network Settings

---

Update:

First, I recommend you disable the ipv6 in your SO, you can do this following these steps:

In /etc/sysctl.conf:

net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1

To disable in the running system:

echo 1 > /proc/sys/net/ipv6/conf/all/disable_ipv6
echo 1 > /proc/sys/net/ipv6/conf/default/disable_ipv6

or

sysctl -w net.ipv6.conf.all.disable_ipv6=1
sysctl -w net.ipv6.conf.default.disable_ipv6=1

After that, you must change in elasticsearch.yml the value of network.bind_host in both nodes with their respective IP's

# Elasticsearch, by default, binds itself to the 0.0.0.0 address, and listens
# on port [9200-9300] for HTTP traffic and on port [9300-9400] for node-to-node
# communication. (the range means that if the port is busy, it will automatically
# try the next port).
# Set the bind address specifically (IPv4 or IPv6):
#
network.bind_host: 10.0.0.1
# Set the address other nodes will use to communicate with this node. If not
# set, it is automatically derived. It must point to an actual IP address.
#
network.publish_host: 10.0.0.1

Or set

# Set both 'bind_host' and 'publish_host':
#
network.host: 10.0.0.1

Finally you must validate the configuration of your network adapters, both must be configured correctly with IP that you used before.

Hope this helps