Despite all the great tools out there these days for monitoring, it’s difficult to get away without writing a custom monitoring script or two. Spools, various types of cache, multi-server indexing. So many things that can go wrong.
Once upon a time I used to write quick and dirty complex scripts that would email in one case, otherwise do some-such-thing, etc. They were brittle with questionable reliability. Talk about a false sense of security.
These days I like to use an untouted feature of mail to keep things simpler. The idea is to only make noise when there is a problem. If all is well, keep silent.
Take this cron job for example:
*/5 * * * * root /somedir/check-spool.sh | mail -E -s 'Spool Monitor' firstname.lastname@example.org
check-spool.sh will do just that: Check a spool each time it is run. If everything is fine, no output. If it finds a problem then it outputs what it finds. The “big E” (
-E) switch tells mail to ignore any body-less input, and will therefore only spam us when there is some panic-worthy output.
Thanks big E.