5

We have a new client with a webserver that's getting too much proxy abuse. The access_log is filled with these kind of lines:

64.187.XXX.XXX - - [12/May/2015:10:32:10 -0300] "GET https://ads.exoclick.com:443/ads.js HTTP/1.0" 404 204 "http://www.salefutures.com/?p=1349" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/4.0.206.1 Safari/532.0"

I obviously thought first about fail2ban to ban via IPtables every offending IP, and yep, it worked, but we're receiving so much requests that fail2ban started to eat RAM like crazy.

Besides that fail2ban works OK banning IP's but the problem is that this webserver is a very resource-limited VM and behind it there's a D-Link firewall DFL-2560G.

The thing is that these type of requests are generating in Layer 7 (as HTTP goes in application layer) and it's proving to be very difficult to filter these kind of requests before they reach the webserver. Additionally, this server's app-filtering module need a licence which is not bought by the client, so it leaves me with very few options, and I really don't know where to start.

Can anyone give me some ideas?

Improve this question
8
  • Why not just add the address in the iptables and permban it?
    – munchkin
    May 21, 2015 at 5:39
  • 1
    Because: 1- I was already doing it with fail2ban, and proved to be very resource-hungry. The list is so enormous that my iptables is going 70K ips and its still growing in number. Consider that this is hosted in a very humble VM. 2- The client, reasonably, wants to filter those GET requests in the firewall.
    – oscillat0r
    May 21, 2015 at 13:34
  • One of the reasons why I don't use Fail2ban is because it's way more complicated than it needs to be and this question demonstrates that. Just roll your own equivalent that only parses the log and adds the IPs in an IPSet without keeping anything in RAM.
    – user42178
    May 21, 2015 at 16:27
  • 1
    If you're on a cloud, you'd be able to provision something to offload the processing. So, shift off logging into another machine, separate interface for internal, bring up a separate proxy etc. However, if you're on one machine, or bare metal, or have cannot bring anything else up, then disable or at least offline the fail2ban and do insertions that don't affect the ram, i.e straight insertions.
    – munchkin
    May 22, 2015 at 8:27
  • 1
    Was your web server ever configured as anonymous reverse proxy? If so it may have ended up in a public proxy list and you may be better off switching IP.
    – Enos D'Andrea
    Jun 1, 2015 at 22:47

1 Answer 1

Reset to default
2

Evaluate mod_security - see https://www.modsecurity.org/about.html.

I don't know off the top of my head if there are rules to help with OP's problem - if there are, of course, use them. If not rules are easily written. Disable rules that are not applicable to reduce the negative performance impact the module will cause.

An additional benefit would be that of the overall Web Application Firewall framework. You may find you need additional protections which Fail2Ban may not help fulfill.

Improve this answer
2
  • can you elaborate your answer, Why use mod_security, Why disable the standard rules, What makes this better than 'other' options.
    – LvB
    Jun 6, 2015 at 2:17
  • Expanded my answer
    – user1801810
    Jun 6, 2015 at 23:09

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .