We have a new client with a webserver that's getting too much proxy abuse. The access_log is filled with these kind of lines:
64.187.XXX.XXX - - [12/May/2015:10:32:10 -0300] "GET https://ads.exoclick.com:443/ads.js HTTP/1.0" 404 204 "http://www.salefutures.com/?p=1349" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/532.0 (KHTML, like Gecko) Chrome/18.104.22.168 Safari/532.0"
I obviously thought first about fail2ban to ban via IPtables every offending IP, and yep, it worked, but we're receiving so much requests that fail2ban started to eat RAM like crazy.
Besides that fail2ban works OK banning IP's but the problem is that this webserver is a very resource-limited VM and behind it there's a D-Link firewall DFL-2560G.
The thing is that these type of requests are generating in Layer 7 (as HTTP goes in application layer) and it's proving to be very difficult to filter these kind of requests before they reach the webserver. Additionally, this server's app-filtering module need a licence which is not bought by the client, so it leaves me with very few options, and I really don't know where to start.
Can anyone give me some ideas?