Universal Log Analyser
1 How it works
This uses a quick and dirty Perl plugin setup where the plugins you wish to use are simply tacked onto the end of the command line. This allows multiple instances of the script to be used, looking at different logs, with different plugins, and storing the results in a different place, or in the same if you prefer (but ensure the script is not run concurrently with the same results file - it will get messy!)
Plugins are simply Perl modules which return a reference to a function to register the plugin. The register functions are run to collect up the stuff that actually does the work inside the plugin, but we will go into plugin internals later.
The script will handle loading and saving of results as well as tracking where we where in the log file(s) and continuing on from the same place which makes the plugins rather simple - they just need to recognise events in the line they are given and count them.
Secondly, basic shell scripts can be used in the snmpd config to find the relevant data and return it for transmission via SNMP. These can either be universal and you specify the data fields on the command line (great for just a few), or treat them more as config files and hard-code the data fields into the script which is probably more manageable when many fields are returned in one query.
dovecot-snmp-cronjob #!/bin/sh /usr/bin/perl /etc/snmp/uloganalyser /var/log/mail.log.1 /var/log/mail.log /var/local/snmp/mail dovecot policyd-spf postfix opendkim # where to keep the files STORE=/var/local/snmp # update smart parameters for devfull in /dev/sd?; do dev=`basename $devfull` /usr/sbin/smartctl -n idle -a $devfull >$STORE/smart-$dev.TMP mv $STORE/smart-$dev.TMP $STORE/smart-$dev done # run fail2ban stats /etc/snmp/uloganalyser \ /var/log/fail2ban.log.1 \ /var/log/fail2ban.log \ /var/local/snmp/fail2ban \ fail2ban #vboxadm spamassissin /etc/snmp/uloganalyser /var/log/vboxadm/sa.log.1.gz /var/log/vboxadm/sa.log /var/local/snmp/sa vboxadm-sa #snmpd needs access chown snmp /var/local/snmp/* #cacti (www-data) needs access chgrp www-data /var/local/snmp/smart* chmod g+r /var/local/snmp/smart*
4 Wired stats
If you get wired stats like this
even you had no "Maximum connections reached" in your logfile then have a look at your cacti logfile:
As you see sometimes cacti gets the value "0" but the real value is 1770.
I think the reason is that cacti-poller and uloganalyser are running at the same time. While uloganalyser is still running, cacti asks already snmpd but /var/local/snmp/mail is not written so value is "0". Also uloganalyser creates a tmp-file
Solution: Start uloganalyser by cron with an offset of 4min
#/etc/cron.d/cacti-dovecot MAILTO=root #http://stackoverflow.com/questions/12786410/run-cron-job-every-n-minutes-plus-offset #this scripts needs to be ready before cacti 4-59/5 * * * * root /etc/snmp/dovecot-snmp-cronjob 2>/dev/null