What is the benefit of combining different web log files with different formats?

Question

What is the benefit of combining different web log files with different formats in terms of web application security? In other words, how does combining iis log file with apache log file help in identifying attacks?

Answer 1

What you are describing in your question is often called Informationfusion. By being able to connect different sources of logs you can more easily increase knowledge of the context of the requests and help to determine its interpretation.

The more logs you can put in context with other logs and information the more valuable they become.

For example say your webservers are all sent to a central logging device where their format is unified. Once your logs now are unified you can quite easily use commands like:

cat weblogs.tz | cut $ip > ip.txt
netcat whois.cymru.org 43 < ip.txt > result.txt

And your now sitting on your users ASN, IP-address, Nettrange, country and other nice stuff from all your webservers. Taking this a step further you now look for suspicious clients who are visiting your servers in patterns that is not normal, maybe looking for a focused attack.

Now with all your webserver logs you could also look more into any patterns that is seen on all or just some of your webservers much easier. Like this list from zeltser.com/log-management/security-incident-log-review-checklist:

• Excessive access attempts to non-existent files
• Code (SQL, HTML) seen as part of the URL
• Access to extensions you have not implemented
• Web service stopped/started/failed messages
• Access to “risky” pages that accept user input
• Look at logs on all servers in the load balancer pool
• Error code 200 on files that are not yours
• Failed user authentication Error code 401, 403
• Invalid request Error code 400 Internal server error Error code 500

Answer 2

If you convert IIS log files to Apache format, you can then run apache-scalp.googlecode.com on them.