Analyzing Apache log with Snort

Question

I need to analyze an Apache log with Snort and others IDS/WAFs (Suricata, mod_security and Shadow Daemon). In order to do so, I was thinking about create TCP packets with the GET and POST requests stored in the Apache log with Scapy in Python. Something like this:

packet= IP(dst=dst_ip)/TCP(dport=9999)/Raw(load=payload) #payload contains the http request

I store this TCP packets into a PCAP file to later, analyze it with Snort or the another IDS/WAFs I said.

The problem with this method of building packets is that there is no state in the communication and Snort detects it with this alert:

[**] [129:2:1] Data on SYN packet [**]
[Classification: Generic Protocol Command Decode] [Priority: 3] 
09/01-20:29:50.816860 127.0.0.1:20 -> 127.0.0.1:9999
TCP TTL:64 TOS:0x0 ID:1 IpLen:20 DgmLen:102
******S* Seq: 0x0  Ack: 0x0  Win: 0x2000  TcpLen: 20
[Xref => http://www.securityfocus.com/bid/34429][Xref => http://cve.mitre.org/cgi-bin/cvename.cgi?name=2009-1157]

Then, I adapted the code to add a sequence and ack number:

ip = IP(src=src_ip, dst=dst_ip)
packet = (ip / TCP(sport=src_port, dport=dest_port, flags='PA',
      seq=seq_n, ack=ack_n) / Raw(load=fullrequest[0])

seq_n = seq_n + len(payload.encode('UTF8'))

In this way, there is a sequence but the Data on SYN packet alert changes for another (although instead of leaving as many alerts as the same number of packages, only 22% of the packets throw an alert):

[**] [129:12:1] Consecutive TCP small segments exceeding threshold [**]
[Classification: Potentially Bad Traffic] [Priority: 2] 
09/01-20:49:15.037299 127.0.0.1:60664 -> 127.0.0.1:80
TCP TTL:64 TOS:0x0 ID:1 IpLen:20 DgmLen:94
***AP*** Seq: 0x156E7  Ack: 0xB  Win: 0x2000  TcpLen: 20

In the end, I chose to create a client-server structure with sockets (sending the payload from one virtual machine to another), analyze the traffic with WireShark and then save the packages as PCAP. The problem here is that Snort does not detect a single attack. In addition, I can not automate this analysis operation.

Attacks example:

"GET /shoutbox.php?conf=../../../../../../../../etc/passwd HTTP/1.1"
"GET /cgi-bin/apexec.pl?etype=odp&template=../../../../../../../../../../etc/hosts%00.html&passurl=/category/ HTTP/1.1"

I am using Snort with Pulledpork to download the rules and I have tried it with a PCAP that I was using in the postgrade (not manually built) and it is detecting attacks. Maybe there is something wrong at the time of creating packets.

Here is my Snort and Pulledpork conf:

Snort: snort.conf

Pulledpork: pulledpork.conf

Here are my PCAPs:

First way (Data on SYN packet): output.pcap

Second way (Consecutive TCP small segments exceeding threshold): output_seq.pcap

What can I be doing wrong? Any hint? Any easier way to detect attacks in an Apache log with IDS/WAF?

Answer 1

Your approach to this is...a bit misguided.

You shouldn't need to be crafting packets to replay events from an Apache log. The log is a list of HTTP requests, so use a library that can make proper HTTP requests, like the requests library itself. You don't even have all the information you need to craft these packets anyway-- in the case of POSTs, you don't know what the payload actually was, only where it was sent.

Configure your IDSes to all monitor the same interface. Given a list of URLs/paths, write a script to simply request each one. Then have each IDS dump its output somewhere and see what each detected. You can repeat this process and evaluate or tune rules as needed.