5

We have a new client with a webserver that's getting too much proxy abuse. The access_log is filled with these kind of lines:

64.187.XXX.XXX - - [12/May/2015:10:32:10 -0300] "GET https://ads.exoclick.com:443/ads.js HTTP/1.0" 404 204 "http://www.salefutures.com/?p=1349" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.206.1 Safari/532.0"

I obviously thought first about fail2ban to ban via IPtables every offending IP, and yep, it worked, but we're receiving so much requests that fail2ban started to eat RAM like crazy.

Besides that fail2ban works OK banning IP's but the problem is that this webserver is a very resource-limited VM and behind it there's a D-Link firewall DFL-2560G.

The thing is that these type of requests are generating in Layer 7 (as HTTP goes in application layer) and it's proving to be very difficult to filter these kind of requests before they reach the webserver. Additionally, this server's app-filtering module need a licence which is not bought by the client, so it leaves me with very few options, and I really don't know where to start.

Can anyone give me some ideas?

8
  • Why not just add the address in the iptables and permban it? – munchkin May 21 '15 at 5:39
  • 1
    Because: 1- I was already doing it with fail2ban, and proved to be very resource-hungry. The list is so enormous that my iptables is going 70K ips and its still growing in number. Consider that this is hosted in a very humble VM. 2- The client, reasonably, wants to filter those GET requests in the firewall. – oscillat0r May 21 '15 at 13:34
  • One of the reasons why I don't use Fail2ban is because it's way more complicated than it needs to be and this question demonstrates that. Just roll your own equivalent that only parses the log and adds the IPs in an IPSet without keeping anything in RAM. – user42178 May 21 '15 at 16:27
  • 1
    If you're on a cloud, you'd be able to provision something to offload the processing. So, shift off logging into another machine, separate interface for internal, bring up a separate proxy etc. However, if you're on one machine, or bare metal, or have cannot bring anything else up, then disable or at least offline the fail2ban and do insertions that don't affect the ram, i.e straight insertions. – munchkin May 22 '15 at 8:27
  • 1
    Was your web server ever configured as anonymous reverse proxy? If so it may have ended up in a public proxy list and you may be better off switching IP. – studio2012 Jun 1 '15 at 22:47
2

Evaluate mod_security - see https://www.modsecurity.org/about.html.

I don't know off the top of my head if there are rules to help with OP's problem - if there are, of course, use them. If not rules are easily written. Disable rules that are not applicable to reduce the negative performance impact the module will cause.

An additional benefit would be that of the overall Web Application Firewall framework. You may find you need additional protections which Fail2Ban may not help fulfill.

2
  • can you elaborate your answer, Why use mod_security, Why disable the standard rules, What makes this better than 'other' options. – LvB Jun 6 '15 at 2:17
  • Expanded my answer – user1801810 Jun 6 '15 at 23:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.