ELK Stack as a SIEM - First steps

Question

I have experience with a couple of commercial SIEM solutions running on Security Operation Centers. I've been reading about companies using Splunk as SIEM. I'm a big fan of open source solutions and I found that the ELK Stack can do the same thing.

I'm trying to set up a ELK Stack to learn more about the technology and possibilities. I have some doubts about integrating syslog-ng with ELK. Is that really necessary?

I assume that the input to the ELK platform will be all syslogs and system events. All correlation, rules and processing happens within the ELK environment. Am I correct?

In terms of syslog, syslog-ng can record events to a DB instead of text files. Is DB a better way to integrate with Elasticsearch?

Thank you for your time and any help is welcome.

Answer 1

I have some doubts about integrating syslog-ng with ELK. Is that really necessary?

Of course not, syslog-ng is a "log management infrastructure." If you deploy ELK as a SIEM you can manage all your logs there. You just need to install an agent on each server you want logs from. You can use the official logstash client or any other tool that parses your syslog to JSON and pass it to the Elastic Search server.

I assume that the input to the ELK platform will be all syslogs and system events. All correlation, rules and processing happens within the ELK environment. Am I correct?

You can pass anything you want, but yes you are correct.

Is DB a better way to integrate with Elasticsearch?

I don't think so; Elastic gets all the data and stores it by itself, so to interact with any other DB you will have to deploy some agent that reads the DB and sends JSON to the Elastic server.

You can follow this guide to start playing with the full ELK: here

I'm not sure it is up to date (for example, Kibana doesn't need nginx anymore for reverse proxy, it comes with built-in nodejs) but it can save you a lot of time.

Answer 2

I'm trying to set up a ELK Stack to learn more about the technology and possibilities. I have some doubts about integrating syslog-ng with ELK. Is that really necessary?

I, along with many others, consolidate syslog to a common location and then ingest the logs from there. It's absolutely possible, but by no means required.

I assume that the input to the ELK platform will be all syslogs and system events. All correlation, rules and processing happens within the ELK environment. Am I correct?

That's what the ELK is for.

I want to step back a second, though.

SIEM solutions

A SIEM generally needs to be more than a base ELK deployment. It generally needs more than basic Splunk deployment, too. There's a reason that folks pay so much for the Enterprise Security add-on for Splunk. A SIEM is no joke -- especially if you're worried about any kind of compliance requirements.

Are you really looking for a bona fide SIEM, or just a SMB SIEM-ish ELK?

Answer 3

While there aren't as many pre-configured items with ELK as there are with Splunk, I feel it's more powerful if you commit yourself to learning the stack.

The real benefit of using elasticsearch is that it does great at searching discrete, unstructured, data sets. You can transform events, logs, and data, with Logstash, like setting common field names across all event types, e.g. [src][ip], [src][port].

The neatest thing that will turn heads of execs is being able to geotag IP addresses and show all your network connections on a map. If you cannot geofence your network, you will want to know where folks are connecting to your network from. Using the geotag filter in your Logstash configuration will allow you to add latitude, longitude, country name, area code, etc. to each event as it goes out to Elasticsearch, with as little as four lines of code.