To enable disk statistics collection/trending, edit /etc/cron.d/sysstat and change the following:

*/10 * * * * root /usr/lib64/sa/sa1 1 1

to this:

*/10 * * * * root /usr/lib64/sa/sa1 -d 1 1

(Obviously, replace “lib64″ with “lib” as appropriate for i386 systems.)

Either wait for the next sa log rotation (at midnight) for sa1 to begin collecting disk statistics, or delete your current day’s statistics. sa1, for whatever historical reason, does not add new counters to an existing sa log file.

I’ve always wanted to be able to monitor things like number of open sockets, page faults, context switches, and other performance counters. Some of them are available through SNMP; others aren’t. The ones that are available aren’t all available by device. I wanted a little bit more detail.

The other problem with SNMP queries is that a Nagios check doesn’t query an average — something that spikes for a minute is not the same as a condition that persists for several minutes or hours. I wanted to leverage the built-in accounting in sysstat to pull together something Nagios can actually make a little bit of sense out of.

Anyway, I went ahead and created a Nagios plugin that will parse the output of sadf (which is a frontend to sa/sar performance counters). You can query multiple counters at a shot, specifying separate alert thresholds for each (or none at all, if you just want performance data). You can specify, via shell-style glob patterns, which devices you want to include or exclude, so that you can, for example, exclude all “lo” and “tun*” devices from network statistic monitoring. You can also pick the sampling period, so if you want an average of the last 30 minutes the plugin will produce it.

You can do stuff like this:

./check_sa.pl -i -C %usr -C %soft -C %sys -C %idle -D all
SA OK – All counters within specified thresholds. | %idle[cpu0]=96.84;; %idle[cpu1]=96.31;; %idle[cpu2]=97.23;; %idle[cpu3]=95.8;; %soft[cpu0]=0;; %soft[cpu1]=0.01;; %soft[cpu2]=0;; %soft[cpu3]=0.01;; %sys[cpu0]=0.4;; %sys[cpu1]=0.46;; %sys[cpu2]=0.36;; %sys[cpu3]=0.63;; %usr[cpu0]=2.67;; %usr[cpu1]=3.13;; %usr[cpu2]=2.27;; %usr[cpu3]=3.46;;

Or, if you prefer to summarize:

./check_sa.pl -i -C %usr -C %soft -C %sys -C %idle -d all
SA OK – All counters within specified thresholds. | %idle[all]=96.54;; %soft[all]=0;; %sys[all]=0.46;; %usr[all]=2.89;;

It’s still a tiny bit slow — it takes about 500-600 ms to run on the systems I’ve tested — but this should be good enough to be useful without bogging down Nagios too badly.

The script requires the Text::Glob module to be installed, so it can convert shell-style globs into regular expressions to match against.

View the project:

• On GitHub
• On MonitoringExchange