Opsview - BSM - Message Queue Count - METRIC CRITICAL - Message Count is ...msgs

Symptoms

• Opsview - BSM - Message Queue Count service check is critical with status info Message Count is ...msgs

Cause: Loss of database connection

Known issue OP-48285

Losing network connectivity to the database can cause the bsm-queue message queue to back up and not clear down—even after the network connection is restored.

Solution: Restart opsview-bsm

Typically, restarting the opsview-bsm component should resolve this issue.

1. Log in to your BSM server as root (this is usually your orchestrator).

2. Restart the opsview-bsm component using opsview-monit:

/opt/opsview/watchdog/bin/opsview-monit restart opsview-bsm

Solution: Event handler to restart opsview-bsm

If you encounter this issue often, it may be best to create an event handler for when this service check goes critical that restarts the opsview-bsm component.

Refer to our event handler documentation for more information about how to do this. Here is an example script you could use to restart the opsview-bsm component:

#!/bin/bash
# If Service State is CRITICAL (options are OK, WARNING, CRITICAL and UNKNOWN)
# and Service State Type is HARD (options are HARD and SOFT)
# then execute Event Handler action
set -e
if [[ "$NAGIOS_SERVICESTATE" = "CRITICAL" && "$NAGIOS_SERVICESTATETYPE" = "HARD" ]]; then
    echo "restarting opsview-bsm"
    /opt/opsview/coreutils/bin/opsview_watchdog opsview-bsm restart >/dev/null 2>&1
    # record event to syslog
    logger "opsview-bsm restarted by Opsview $NAGIOS_HOSTADDRESS"
fi