Universal Log Analyser
How it works
This uses a quick and dirty Perl plugin setup where the plugins you wish to use are simply tacked onto the end of the command line. This allows multiple instances of the script to be used, looking at different logs, with different plugins, and storing the results in a different place, or in the same if you prefer (but ensure the script is not run concurrently with the same results file - it will get messy!)
Plugins are simply Perl modules which return a reference to a function to register the plugin. The register functions are run to collect up the stuff that actually does the work inside the plugin, but we will go into plugin internals later.
The script will handle loading and saving of results as well as tracking where we where in the log file(s) and continuing on from the same place which makes the plugins rather simple - they just need to recognise events in the line they are given and count them.
Secondly, basic shell scripts can be used in the snmpd config to find the relevant data and return it for transmission via SNMP. These can either be universal and you specify the data fields on the command line (great for just a few), or treat them more as config files and hard-code the data fields into the script which is probably more manageable when many fields are returned in one query.
Plugins
Cronjobs
dovecot-snmp-cronjob
#!/bin/sh
/usr/bin/perl /etc/snmp/uloganalyser /var/log/mail.log.1 /var/log/mail.log /var/local/snmp/mail dovecot policyd-spf postfix opendkim
# where to keep the files
STORE=/var/local/snmp
# update smart parameters
for devfull in /dev/sd?; do
dev=`basename $devfull`
/usr/sbin/smartctl -n idle -a $devfull >$STORE/smart-$dev.TMP
mv $STORE/smart-$dev.TMP $STORE/smart-$dev
done
# run fail2ban stats
/etc/snmp/uloganalyser \
/var/log/fail2ban.log.1 \
/var/log/fail2ban.log \
/var/local/snmp/fail2ban \
fail2ban
#vboxadm spamassissin
/etc/snmp/uloganalyser /var/log/vboxadm/sa.log.1.gz /var/log/vboxadm/sa.log /var/local/snmp/sa vboxadm-sa
#snmpd needs access
chown snmp /var/local/snmp/*
#cacti (www-data) needs access
chgrp www-data /var/local/snmp/smart*
chmod g+r /var/local/snmp/smart*
Wired stats
If you get wired stats like this
even you had no "Maximum connections reached" in your logfile then have a look at your cacti logfile:
As you see sometimes cacti gets the value "0" but the real value is 1770.
I think the reason is that cacti-poller and uloganalyser are running at the same time. While uloganalyser is still running, cacti asks already snmpd but /var/local/snmp/mail is not written so value is "0". Also uloganalyser creates a tmp-file
Solution: Start uloganalyser by cron with an offset of 4min
#/etc/cron.d/cacti-dovecot
MAILTO=root
#http://stackoverflow.com/questions/12786410/run-cron-job-every-n-minutes-plus-offset
#this scripts needs to be ready before cacti
4-59/5 * * * * root /etc/snmp/dovecot-snmp-cronjob 2>/dev/null