I occasionally get clients requesting I look at their access_log file to determine if any web attacks were successful. What tools are helpful to discern attacks?
Yes you can, apache log gives you information about people who visited your website including bots and spiders. patterns you can check:
- someone made multiple requests in less than second or accepted time frame.
- accessed secure or login page multiple times in a one minute window.
- accessed non existent pages using different query parameters or path.
apache scalp http://code.google.com/p/apache-scalp/ is very good at doing all the above things
As Ams noted, log analysis won't cover all attacks and you won't see parameters of POST requests. However, analyzing logs for POST requests sometimes is very rewarding.
Specifically, POSTs are popular for sending malicious code to backdoor scripts. Such backdoors can be created somewhere deep in subdirectories or a backdoor code can be injected into a legitimate file. If your site is not under a version control or some other integrity control, it may be hard to locate such backdoor scripts.
Here's the trick:
- Scan your access logs for POST request and compile a list of requested files. On regular sites, there shouldn't be many of them.
- Check those files for integrity and legitimacy. This will be your white list.
- Now regularly scan your logs for POST request and compare requested files with your white list (needless to say you should automatize this process). Any new file should be investigated. If it is legitimate - add it to the whitelist. If not - investigate the problem.
This way you'll be able to efficiently detect suspicious POST request to files that normally don't accept POST requests (injected backdoor code) and newly created backdoor files. If you are lucky, you can use the IP address of such requests to identify the initial point of penetration or you can simply check log around that time for suspicious activity.
mod_sec can detect just about anything, including inspection of POST requests.
You could even load up snort ids rules into it and block these requests on the fly before they hit applications
apache-scalp can check for attacks via HTTP/GET:
"Scalp! is a log analyzer for the Apache web server that aims to look for security problems. The main idea is to look through huge log files and extract the possible attacks that have been sent through HTTP/GET"
Check out WebForensik
It's a PHPIDS-based script (released under GPL2) to scan your HTTPD logfiles for attacks against web applications.
- supports standard log formats (common, combined) - allows user-defined (mod_log_config syntax) formats - automatically pipes your web logs through PHPIDS - categorizes all incidents by type, impact, date, host... - generates reports in CSV, HTML (sortable table), XML
It may be better to scan your database plan cache (and/or log files) than your web server logs, although certainly it would be good to combine these techniques and match up time and date stamps.
For more information, please see the book by Kevvie Fowler on SQL Server Forensic Analysis.
Try LORG -> https://github.com/jensvoid/lorg. It has different detection modes (signature-based, statistics-based, learning-based), some nice features like geomapping, DNSBL-lookups and robot detection (= was the attacker a man or a machine?).
It can make a guess on the success of attacks by looking for outliers in the 'bytes-sent' field, HTTP response codes or active replay of attacks.
Code is still pre-alpha, but under active developement.