How do you handle log analysis in your business?

I want to establish a good way of finding exceptions and alerts in the server logs, either automatically or semi automatic (having to review exceptions and alerts).

Is this functionality that is usually handled by an IDS/IPS?

Right now I am collecting all log data from web servers, databases, syslog and such into Splunk, and I'm considering to make Splunk dashboards which presents me weird and non-normal log entries. Is this the right way to go?

  • How large is your organization, and what kind of volume are you talking about?
    – Scott Pack
    Commented Dec 8, 2010 at 13:46
  • You can consider this organization a medium sized business. 300-400 employees.
    – Chris Dale
    Commented Dec 16, 2010 at 11:01

5 Answers 5


OSSEC is a great FOSS HIDS that works well to reduce the number of log events you need to check out to something potentially manageable.

It supports all of the log sources you mentioned: http://www.ossec.net/main/supported-systems

We built an in-house app that allows us to do a quick “daily check” of all the alerts OSSEC fires in a day; it typically takes us less than 20 minutes of time to call out an all clear or to kick-off an incident investigation. For one environment we receive ~70,000 OSSEC alerts per day, but most of the alerts/events are clusters of auth events or app errors spamming the logging system, for example:

alt text

Since you have Splunk going you could do the Splunk + OSSEC integration: http://www.ossec.net/main/splunk-ossec-integration

The other day I saw a couple enterprise SIEMs in action – I was stunned to learn they apparently have no canned reports detailing unusual or rare log events. Unusual and rare events are frequently the best indicators of something bad and can serve as your short list of events to chase down until the leads turn cold.

We also use Logwatch as a secondary log analyzer.

  • Can you elaborate on which SIEM's you saw that didn't have canned reports ? To the best of my knowledge reporting has substantially improved of late.
    – Geek
    Commented Dec 8, 2010 at 11:44
  • One was RSA enVision and I believe the other was LogRhythm (I was specifically asking the engineers if they had any built-in reports that would detail unusual or rare events). Commented Dec 8, 2010 at 17:08
  • Try ArcSight, Loglogic or CAELM. I think IBM has a product also for the same.
    – Geek
    Commented Dec 9, 2010 at 13:37

This functionality is generally achieved with the help of SIM or SIEM solutions. There are several popular vendors for these CA, ArcSight, Loglogic, splunk etc.

You can collect the Logs, be informed about the Alerts and take action based on Correlation of rules. You can collect logs from all possible sources, Databases, Unix, Windows, Firewalls etc.


Methods often differ depending on scale and location, for example perimeter logging for an enterprise is usually too much for an internal alert team to cope with, so this is usually outsourced to one of the managed service suppliers, who will then pass on relevant alerts.

For smaller organisations, or for specific host logging the problem is much more manageable, however you are still going to have to look at how you train and tune the alerting process. Splunk and others will do the job, but you need to plan for the amount of resource required to look after during the first few weeks, on any upgrade or change to the logged environment, on any change in attack profiles, and in fact continuously over time.

Having that out the way, the usual mechanisms are to use both signature based and statistical alerting - signatures are fast and can be updated by service providers so require low effort, however they need to be created so typically do not identify new types of attack, whereas statistical alerting responds to any change in logging - so you can end up overloaded with false positives until you tune.


From a security viewpoint (there's lots of other things which are worth checking logs for) then the logs will only reliably show where the security is working as expected. If someone has bypassed the security measures then it probably won't be detected by the logs / the logs may be compromised - so they are of limited use.

Also, if you can construct mechanism to analyse the logs for potential violations, then, in most cases it's just as easy to enforce the policy rather than wait for a violation to occur.

Log analysis is no substitute for a host-based file integrity checker (such as Lids, tripwire)

Having said that, I would recommend using fail2ban where appropriate.


Also the log analysis should take into account the organisation's change management routine - or more specifically the software/hardware update process - at least in the sense of whether file integrity has changed.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .