Cassandra nodetool connection timed out

Question

Im trying to use nodetool to check the status of my cluster, but its unable to connect.

My cassandra.yaml is configured with listen_address and rpc_address set as the server IP (e.g. 10.10.10.266).

Im able to connect through cqlsh and cassandra-cli using the same IP, but when I connect to nodetool it doesnt work.

/bin$ nodetool -h 10.10.10.266 ring
Failed to connect to '10.10.10.266:7199': Connection has timed out

I dont think I have a firewall enabled on the server (Ubuntu). Im running this directly on the server in question, so I wouldnt have thought it would be a firewall issue anyway.

Answer 1

You probably need to uncomment the following parameter in cassandra-env.sh:

-Djava.rmi.server.hostname=<public name>

Replace with the address of the interface you want the jmx interface to listen on.

Answer 2

nodetool connects through JMX interface. By default it's listening on port 7199 (other tools use RPC interface listening on port 9160 by default). Check JMX settings in cassandra-env.sh file. Most likely JMX server is listening on wrong interface (or probably loopback interface).

Default JMX configuration section (cassandra ver. 1.1.5) contains link to troubleshooting guide:

# jmx: metrics and administration interface
# 
# add this if you're having trouble connecting:
# JVM_OPTS="$JVM_OPTS -Djava.rmi.server.hostname=<public name>"
# 
# see 
# https://blogs.oracle.com/jmxetc/entry/troubleshooting_connection_problems_in_jconsole
# for more on configuring JMX through firewalls, etc. (Short version:
# get it working with no firewall first.)
JVM_OPTS="$JVM_OPTS -Dcom.sun.management.jmxremote.port=$JMX_PORT" 
JVM_OPTS="$JVM_OPTS -Dcom.sun.management.jmxremote.ssl=false" 
JVM_OPTS="$JVM_OPTS -Dcom.sun.management.jmxremote.authenticate=false" 
JVM_OPTS="$JVM_OPTS $JVM_EXTRA_OPTS"

It also worths to list all network interfaces using ifconfig and try telnet'ing port 7199 on all interfaces.