I am studying SIEM tools.
Firewall logs will be different from IDS logs and even from Antivirus logs.
How can log aggregation take place?
Information Security Stack Exchange is a question and answer site for information security professionals. It only takes a minute to sign up.Sign up to join this community
The logs do not need to be the same. They do not even need to log data about the same types of information (although it can help). Aggregation is simply to gather the information together.
In a SIEM, what you are most concerned with are identifiable entities: users, IPs, domains, services, etc. Firewall, IDS, and Antivirus logs will contain information about these things, and that's how a SIEM can stitch together information.
The logic in the aggregator pulls out the data fields to provide a timeline and a logic flow:
The logs themselves do not need to record the same information or even be in the same format. The data fields just have to be recognisable enough for the log aggregator to identify and stitch the data points together.
The way logs are aggregated/stored depends on the use-case which you have deployed your SIEM and the deployment architecture.
A typical SIEM will offer you following use cases:
The deployment scenario could be also varied depending on the scale/scope of deployment and the way logs are aggregated also will depend on these scenarios. For example, the following could be some scenarios:
Deploying Log Collectors on the source locations and central log server to store logs. Then send only security-related logs to SIEM database - only the required logs will be stored in SIEM database and rest will be in the log central log server
Collect, store and process entire logs in SIEM database - where the entire logs will be stored in SIEM
A SIEM collecting logs and processing is unique to each SIEM supplier as most of them have their own Connectors/Templates (or supported log formats) for the commonly available devices/log sources. All SIEM vendors will provide you with a list of supported devices and any device not in this list to be connected manually, where the SIEM supplier might help you in drafting your own connector.
You may refer to the following links for a sample SIEM vendor supported device list (connectors are readily available):
Most of the SIEM suppliers have their own way of storing logs (data format to table structure may differ). Most of them will process the received logs and convert to the structure their core platform can understand.
The above-mentioned connectors/templates are used to convert the native source format to the SIEM understandable format.