failed to add munin node to monitoring

Question

I'm trying to setup some new hosts in munin for monitoring. For some reason it ain't happening!

Here's what I've tried so far.

On the munin server, which is already monitoring several other hosts, I've added the host I want in /etc/munin/munin.conf

[db1]
    address   10.10.10.25 # <- obscured the real IP address 
    use_node_name yes

And on the db1 host I have this set in /etc/munin/munin-node.conf

host_name  db1.example.com
allow ^127\.0\.0\.1$
allow ^10\.10\.10\.26$
allow ^::1$
port 4949

And I made sure to restart the services on both machines.

From the monitoring host I can telnet to the new server I want to monitor on the munin port:

[root@monitor3:~] #telnet db1.example.com 4949
Trying 10.10.10.26...
Connected to db1.example.com.
Escape character is '^]'.
# munin node at db1.example.com

Wait a few minutes.. and nothing! The new server won't appear in the munin dashboard on the munin monitoring host.

In the /var/log/munin/munin-update.log log on the db1 host (the one I'm trying to monitor) I find this:

2015/11/30 03:20:02 [INFO] starting work in 14199 for db1/10.10.10.26:4949.

2015/11/30 03:20:02 [FATAL] Socket read from db1 failed.  Terminating process. at /usr/share/perl5/vendor_perl/Munin/Master/UpdateWorker.pm line 254.

2015/11/30 03:20:02 [ERROR] Munin::Master::UpdateWorker<db1;db1> died with '[FATAL] Socket read from db1 failed.  Terminating process. at /usr/share/perl5/vendor_perl/Munin/Master/UpdateWorker.pm line 254.

What could be going on here? And how can I solve this ?

Answer 1

Since you have already verified that your network connection is ok, as a first step of investigation, I would surely simplify the munin-node.conf. Currently you have:

host_name  db1.example.com
allow ^127\.0\.0\.1$
allow ^10\.10\.10\.26$
allow ^::1$
port 4949

From these I would remove:

• host_name (it is probably redundant.)
• The IPv6 loopback address. (I don't think you need it, but you can add it back later if you do need it)
• The IPv4 loopback address. (same as above)

If it still not working, you could completely outrule any issue with the allow config by replacing the direct IPs with:

cidr_allow 10.10.10.0/24

This would allow connection from a full range of IPs in case your db1 host appears to be connecting from a different IP.