Logs are our bread and butter. They store details on everything that happens on any server; each request to each asset on this webpage is logged, every login and email sent. In the case of an outage logs are indispensable to see what happened. If you’re under attack it will be logged. Everything is logged so it is essential to pay attention.
Manually checking all server logs is a slow and arduous task and quickly becomes impossible as you scale up. We actively monitor server logs with Logcheck. Logcheck makes this log monitoring possible across hundreds of servers by reducing the logs needing to be looked at, it does this with the exception tracking approach.
Most log management tools use a blacklist approach, looking for bad words such as “attack”, “bad” and “error”. In doing so they only tell you about the “known bad”, the log lines that have shown errors before. Big problems will come if you’re hosting a brand new app or using new software, there is no way of knowing what is bad and what should be alerted on. You rely completely on the new software to have configured logging that matches your current rules.
Logcheck’s whitelist approach fixes the problem these other tools have, as it passes you all unknown/rogue logs. This makes it perfect for any venture in to the unknown by telling you exceptions to known good rules.
Regex can be used in the whitelist making the rules very customisable and still broad enough to not have to whitelist every single combination of log. We maintain our whitelist rules on a per-server basis, as logs that are OK on one server could indicate a problem on another.
Logcheck and log administration are services offered in our maintenance packages.