Take the 2-minute tour ×
Information Security Stack Exchange is a question and answer site for Information security professionals. It's 100% free, no registration required.

As everyone knows each company wants to save as much money as possible. I have been tasked with pre-filtering logs on less important status/health messages before they get to the SIEM.

  1. Does anyone see the risks involved when pre-filtering before going to SIEM?
  2. Why would someone not do this and or is there a benefit of doing so if you have the resources?
  3. Saving money on per log basis is that worth pre-filtering before it gets to the SIEM?
share|improve this question
add comment

2 Answers

up vote 7 down vote accepted

There are two reasons why pre-filtering is done: to save on the EPS (Events Per Second) rate and to reduce the noise. Every SIEM technology has a maximum EPS analysis capability which are restricted either by the hardware or by the software license. When events exceed that limitation, the SIEM server begin to drop the packets. That is why you would only want to bring in only those events which have significant security implications.

But even more important than the EPS is to reduce the events so that your analyst(s) can analyse those events manually. Although SIEM technology can automate many event correlation activities, initially you need an analyst to go through each of the received log or traffic event manually in order to determine whether it is true positive or false positive. When you overwhelm the system with a lot of noisy events that do not have a security implication, not only are you burdening the system hardware and software resources but you are actually over burdening the analyst in analysing those events as well.

Based on my experience, you need to understand that an effective SIEM solution is the one which are analysing only security events. Many a times people want to let the SIEM analyse and co-relate many non security events as well. At times that might be useful such as correlating the system health information with the received traffic logs in case of a denial of service attacks, however, in most of the cases pre-filtering all non security logs and events and forwarding only the relevant logs to your SIEM will make the system as well as the SIEM analyst more efficient.

share|improve this answer
    
OK great. I wanted to make sure This was an understood method and on path to making a robust SOC. Thank you for your input! –  Jon_Little-Sec Engineer Aug 22 '13 at 23:31
add comment

I work for the company that is developing SIEM solution for last 5 years, so I constantly have to talk to new and existing clients, helping them pick best way to integrate what we offer.

What comes to my mind about SIEM and pre-filtering is if you need to keep your logs intact when entering system because of various country regulations. This might or might not be important for your company.

Secondly, if your SIEM offers $$ plan that concentrates on stored data and not data transferred, you might filter what actually enters storage (retention level, pre-indexing part) so you don't spend too much storage space and you dont kill SIEM alerting with too much junk data. Configuring what enters SIEM and what is discarded at SIEM level and not on OS level gains you more control on what happens if someone quits, you change OS or similar. Look at SIEM rules as documentation for your infrastructure. If it is all in one place it is easier to look at then if its distributed at non standard places.

TLDR I would collect everything and make few rules in retention level so data does not enter indexing level (alerting, correlation etc). If there are regulatory compliances you must follow i would just leave like it as is.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.