Opsview - SNMP traps dropped by some collectors in a cluster

By default, SNMP trap daemons run on all collectors in a cluster. However, only one of these collectors is actually able to receive and process SNMP traps. This collector is the called the master trap node.

If your cluster only has one collector, the fact that only one collector can process SNMP traps is not an issue. However, if your cluster has multiple collectors, SNMP traps may get silently dropped when they are sent to a collector other than the master trap node. This problem worsens as the size of your cluster grows: each collector in the cluster reduces the chances of a trap being sent to the master trap node capable of processing it.

Solution

Limiting the number of collectors running the SNMP traps daemon to two:

• One collector is the master trap node
• The other collector is the standby node

1. Log in to the Opsview host as root.

2. Open the file /opt/opsview/deploy/etc/opsview_deploy.yml in your preferred text editor and locate the entry for the collector cluster you want to change.

3. For each collector you do not want to process SNMP traps, define this variable in its vars: opsview_collector_enable_snmp: False.

We set this variable to False because the SNMP traps daemon is installed by default.

Only collector-01 and collector-02 run the SNMP traps daemon in the below example. The other collectors do not run the daemon because we define the variable opsview_collector_enable_snmp: False in their vars sections.

collector_clusters:
  trap_cluster:
    collector_hosts:
      collector-01: { ip: 192.168.18.53,  ssh_user: centos }
      collector-02: { ip: 192.168.18.157, ssh_user: centos }
      collector-03: { ip: 192.168.18.155, ssh_user: centos, vars: { opsview_collector_enable_snmp: False } }
      collector-04: { ip: 192.168.18.61,  ssh_user: centos, vars: { opsview_collector_enable_snmp: False } }
      collector-05:
        ip: 192.168.18.61
        ssh_user: centos
        vars:
          opsview_collector_enable_snmp: False

After these changes to user_vars.yml, opsview-deploy will not install the SNMP traps daemon on any collectors for which the variable opsview_collector_enable_snmp: False is defined.

4. If you are configuring a new cluster before deployment, stop here. If you are changing an existing cluster, continue to the next step.

5. Log in to all inactive collectors as root.

6. Uninstall the opsview-snmptraps-base and opsview-snmptraps-collector packages from each inactive collector.

CentOS/RHEL

yum remove opsview-snmptraps-base opsview-snmptraps-collector

Ubuntu/Debian

apt-get remove opsview-snmptraps-base opsview-snmptraps-collector

7. Log in to both active collectors as root.
8. Restart the opsview-snmptrapscollector component using opsview-monit.

/opt/opsview/watchdog/bin/opsview-monit restart opsview-snmptrapscollector

9. Restart the opsview-snmptraps component using opsview-monit.

/opt/opsview/watchdog/bin/opsview-monit restart opsview-snmptraps 

Restarting these components forces the collectors to re-elect the master trap node.