How IDS and Firewall Logs are aggregated and feed aggregated log to SIEM?

Question

I am studying SIEM tools.

Firewall logs will be different from IDS logs and even from Antivirus logs.

How can log aggregation take place?

Answer 1

The logs do not need to be the same. They do not even need to log data about the same types of information (although it can help). Aggregation is simply to gather the information together.

In a SIEM, what you are most concerned with are identifiable entities: users, IPs, domains, services, etc. Firewall, IDS, and Antivirus logs will contain information about these things, and that's how a SIEM can stitch together information.

For example:

• IDS log: Known malicious traffic pattern from Source IP 10.10.10.9 to Destination IP 10.10.10.10 at 12:00 pm
• Antivirus log: User Sun-IT disabled Antivirus scanning on IP 10.10.10.10 11:50 am
• Firewall log: Source IP 10.10.10.10 sent a high volume of data out of the network at 12:05 pm

The logic in the aggregator pulls out the data fields to provide a timeline and a logic flow:

• User Sun-IT disabled anti-virus on 10.10.10.10, which was then sent malicious content from 10.10.10.9, and then 10.10.10.10 sent large amounts of data outside of the organisation.
• 10.10.10.9 and Sun-IT are the suspicious actors (a decision support step)
• quarantine 10.10.10.9, 10.10.10.10, reset Sun-IT's credentials and forensically investigate 10.10.10.9 and Sun-IT (this is an automation/orchestration step)

The logs themselves do not need to record the same information or even be in the same format. The data fields just have to be recognisable enough for the log aggregator to identify and stitch the data points together.

Answer 2

The way logs are aggregated/stored depends on the use-case which you have deployed your SIEM and the deployment architecture.

A typical SIEM will offer you following use cases:

1. Log Collection
2. Log Retention
3. Log Analysis
4. Event Correlation
5. Forensics
6. IT Compliance
7. Real-time Alerting
8. User Activity Monitoring
9. File Integrity Monitoring
10. Etc.

The deployment scenario could be also varied depending on the scale/scope of deployment and the way logs are aggregated also will depend on these scenarios. For example, the following could be some scenarios:

• Deploying Log Collectors on the source locations and central log server to store logs. Then send only security-related logs to SIEM database - only the required logs will be stored in SIEM database and rest will be in the log central log server

• Collect, store and process entire logs in SIEM database - where the entire logs will be stored in SIEM

A SIEM collecting logs and processing is unique to each SIEM supplier as most of them have their own Connectors/Templates (or supported log formats) for the commonly available devices/log sources. All SIEM vendors will provide you with a list of supported devices and any device not in this list to be connected manually, where the SIEM supplier might help you in drafting your own connector.

You may refer to the following links for a sample SIEM vendor supported device list (connectors are readily available):

https://www.scribd.com/document/60264371/LogRhythm-Supported-Products-List

Most of the SIEM suppliers have their own way of storing logs (data format to table structure may differ). Most of them will process the received logs and convert to the structure their core platform can understand.

The above-mentioned connectors/templates are used to convert the native source format to the SIEM understandable format.