OP5 Monitor - How to use check_by_snmp_memory on EL7.7

Preface

On Linux systems, there are a few different memory metrics. Free, on a Linux system, means that the RAM is currently completely unused. However, the Linux kernel makes heavily use of cached and buffered memory, and that memory, while not unused, is readily available for applications to use. In most cases, you would therefore be interested in the amount of memory which is available, rather than free. You can estimate how much memory is used and is not available by the following calculation: total_ram - free_ram - buffered_ram - cached_ram. The buffered and cached memory is subtracted, as this memory is available to be used by applications.

Problem

net-snmp has traditionally reported the free metric, and therefore the check plugin check_by_snmp_memory used the above calculation to estimate the amount of used RAM on a Linux system. However, in version 7.7 of RedHat Enterprise Linux and CentOS, the net-snmp package was updated and to report the available memory. As a result, the check plugin no longer reported the correct value, as the buffered and cached memory was included twice.

Update: in EL7.8 they have reverted this, and the -S flag is no longer needed! Admins will manually have to remove the -S flag from hosts updating from EL7.7 to EL7.8!

This is not a problem with the upstream version of net-snmp, or versions of the snmp agents shipped with other operating systems.

Solution

Starting with OP5 Monitor 7.5.4 and 8.0.8, a new -S flag is available to use with the check_by_snmp_memory check plugin. This flag disables the plugins subtraction of buffered and cached memory. To correctly estimate the memory used on Red Hat Enterprise Linux 7.7+ and CentOS 7.7+, edit the check_command to include the -S flag.

This flag is not available in earlier versions of OP5 Monitor and thus an update is needed to fix this problem on earlier versions.

Sometimes, the plugin can detect if the -S flag should have been used with the target system. The plugin will then report an UNKNOWN status, with a message telling the user to use the -S flag. In these cases, please change the check_command to include the -S flag.

All other versions of net-snmp will not have to use the -S flag as of now unless your distribution uses the RedHat version.

Note

In OP5 Monitor version 7.5.3 and 8.0.7, the check_by_snmp_memory plugin was updated to detect whether subtraction of buffered and cached memory should be done. However, this detection did not correctly work on all target systems, resulting in the plugin reporting free memory rather than the available. As it is not possible to detect this with high enough precision, a user defined flag is available as described above, in OP5 Monitor version 7.5.4 and 8.0.8.