Accessing Munin nodes through SSH tunnels

Attention: Content of this page should be moved to the Munin-Guide --> Visit the Guide now.

Starting with munin 2.0 alpha there is native SSH support in the munin-master. This is accompanied with new utilities to make it work in 2.0. To make a 2.0 talk to a earlier version munin-node you can use pmmn. Read more about it in the native ssh article.

Ssh with pre 2.0 Munins

When there is a need to reach one or more hosts behind a non-routing *nix system, you may use SSH tunneling. In this configuration, SSH is used for its convenient tunneling, not for encryption. An alternative to using SSH is bouncing through (x)inetd.

Considering the illustration, an SSH connection is established between the Munin server and the in-between system (referred to as 'bouncer'). This connection will tunnel a number of forwarded ports between the local Munin server and the different Munin nodes. The connection will be encrypted between the Munin server and the bouncer, but unencrypted from the bouncer to the Munin nodes.

Create the SSH connection with the following command:

ssh -L 5001:Server-A:4949 -L 5002:Server-B:4949 -L 5003:Server-C:4949 user@bouncer

Please read the FAQ entry for useful SSH options'''

Then, munin.conf must be configured accordingly:

[localhost]                    # mark localhost with comments
  address            # this hides for munin-node check        
  port 4949                    #
  use_node_name yes            #

# use real hostnames in []

  port 5001
  use_node_name yes

  port 5002
  use_node_name yes

  port 5003
  use_node_name yes

If you have many tunnels to a bunch of servers in one or more DMZs you may want a mechanism, that takes care that the tunnel is up and running. Even if a tunnel is running yet, the connection can drop some day (a short-term network problem, a reboot of the munin-node etc). In this case, this machnism must restart the tunnel soon. Best way to implement this is to write a script, that parse the munin.conf to check if every tunnel is alive or not. The most easy way is an additional section in the /etc/init./munin if the munin-server also run a munin-node.

        # Start or restart ssh tunnel to munin clients in a DMZ
        # File with the munin configuration
        # IP-Address for localhost (local tunnel endpoint)
        # tempoary list

        cat $MUNINCONF  | grep -v "#" | grep "address $IP" --before-context=1 --after-context=1 > $MUNINLIST
        # set an 'end of file' marker
        echo "--" >> $MUNINLIST

        cat $MUNINLIST | while read line ; do
                if [ "$line" != "--" ] ; then
                        if [ "$COUNTER" = "0" ] ; then
                                CLIENTNAME=`echo $line | tr -d '[] '
                        if [ "`echo $line | grep port`" ] ; then
                                PORT=`echo $line | awk '{ print $2 }'`
                        #echo "looking for $CLIENTNAME : $PORT"
                        COUNTER=`echo "$COUNTER + 1" | bc`
                        echo "done with this triple, check for tunnel $CLIENTNAME port $PORT"
                        if [ "`ps -eaf | grep ssh | grep nNgf | grep $PORT | grep -v grep`" ] ; then
                                echo "tunnel to $CLIENTNAME port $PORT is up"
                                echo "SSH tunnel ${CLIENTNAME} NOT alive ... Restarting ..."
                                /usr/bin/ssh -nNgf -L $PORT:localhost:4949 root@$CLIENTNAME 
                                logger -p daemon.notice "SSH tunnel ${CLIENTNAME} NOT alive ... Restarting ...
                                sleep 1

If this script is called with the switch 'check' every tunnel is checkt and restarted if necessary :

done with this triple, check for tunnel Server-A port 5001
tunnel to Server-A port 5012 is up

done with this triple, check for Server-B port 5002
SSH tunnel Server-B NOT alive ... Restarting ...

done with this triple, check for tunnel Server-C port 5003
tunnel to Server-C port 5022 is up

Please take care, that the munin.conf follow the rules given in the example above: first the name (in brackets), then the address (the IP must match the value for the environment variable "IP-Address" in the check script. In the next line the port that should be used. The last thing you need is a cronjob, that runs every hour or more often. This cronjob just calls '/etc/init.d/munin check'.

Last modified at 2016-10-17T02:24:12+02:00 Last modified on 2016-10-17T02:24:12+02:00