18

I am being attacked right now from Tor nodes which are doing 404 requests to my HTTP server. It is from one IP but when i use the DROP iptables rule, it starts again from another IP in a matter of seconds.

It started to ask requests to the cgi-bin folder, but now it is doing requests in my images folder. So based on my investigation it is a tool connected to TOR looking for something and right now it drives me mad.

I tried fail2ban, mod_evasive and mod_security, but those programs get triggered when the other side is looking for one thing or banging at the door at one port. But this tool is looking at every request for a different file.

45.63.100.91 - - [26/Dec/2016:03:27:17 +0100] "HEAD //images/imgsupport.php HTTP/1.1" 404 - "-" "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; MATBJS; rv:11.0) like Gecko"
45.63.100.91 - - [26/Dec/2016:03:27:17 +0100] "HEAD //images/imgppexg.php HTTP/1.1" 404 - "-" "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; MATBJS; rv:11.0) like Gecko"
45.63.100.91 - - [26/Dec/2016:03:27:17 +0100] "HEAD //images/imgppantivirussoft.php HTTP/1.1" 404 - "-" "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; MATBJS; rv:11.0) like Gecko"
45.63.100.91 - - [26/Dec/2016:03:27:17 +0100] "HEAD //images/imgwin95.php HTTP/1.1" 404 - "-" "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; MATBJS; rv:11.0) like Gecko"
45.63.100.91 - - [26/Dec/2016:03:27:17 +0100] "HEAD //images/imgnws.php HTTP/1.1" 404 - "-" "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; MATBJS; rv:11.0) like Gecko"
45.63.100.91 - - [26/Dec/2016:03:27:17 +0100] "HEAD //images/imgroundcorner.php HTTP/1.1" 404 - "-" "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; MATBJS; rv:11.0) like Gecko"
45.63.100.91 - - [26/Dec/2016:03:27:17 +0100] "HEAD //images/imgppcdl.php HTTP/1.1" 404 - "-" "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; MATBJS; rv:11.0) like Gecko"
45.63.100.91 - - [26/Dec/2016:03:27:17 +0100] "HEAD //images/yellowbuy3.php HTTP/1.1" 404 - "-" "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; MATBJS; rv:11.0) like Gecko"
45.63.100.91 - - [26/Dec/2016:03:27:17 +0100] "HEAD //images/imgvirusinfo.php HTTP/1.1" 404 - "-" "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; MATBJS; rv:11.0) like Gecko"
45.63.100.91 - - [26/Dec/2016:03:27:17 +0100] "HEAD //images/escan4.php HTTP/1.1" 404 - "-" "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; MATBJS; rv:11.0) like Gecko"
45.63.100.91 - - [26/Dec/2016:03:27:17 +0100] "HEAD //images/imgredline.php HTTP/1.1" 404 - "-" "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; MATBJS; rv:11.0) like Gecko"
45.63.100.91 - - [26/Dec/2016:03:27:17 +0100] "HEAD //images/imgonlinescan.php HTTP/1.1" 404 - "-" "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; MATBJS; rv:11.0) like Gecko"
45.63.100.91 - - [26/Dec/2016:03:27:17 +0100] "HEAD //images/ram1.php HTTP/1.1" 404 - "-" "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; MATBJS; rv:11.0) like Gecko"
45.63.100.91 - - [26/Dec/2016:03:27:17 +0100] "HEAD //images/imgmanualscan.php HTTP/1.1" 404 - "-" "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; MATBJS; rv:11.0) like Gecko"
45.63.100.91 - - [26/Dec/2016:03:27:17 +0100] "HEAD //images/userdefine.php HTTP/1.1" 404 - "-" "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; MATBJS; rv:11.0) like Gecko"
45.63.100.91 - - [26/Dec/2016:03:27:17 +0100] "HEAD //images/betterinterface.php HTTP/1.1" 404 - "-" "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; MATBJS; rv:11.0) like Gecko"

You see it is looking in the folder images for a specific .php file, but at every request it is looking for a different php file. So, what the hell is this thing doing?

This also eats up my bandwidth, so I am really desperate what I need to do now. Anyone an idea how I can block this?

Server setup: CentOS 7 (OpenVZ, so I am stuck with a kernel where ipset isn't working) with Apache.

  • Try upgrading ipset to 6.30 directly from netfilter.org. See if it is the kernel or ipset that is actually broken. – cybernard Dec 26 '16 at 5:03
  • Are you getting similar requests from other IP addresses? This looks suspiciously like botnet behavior – Henry F Dec 26 '16 at 5:43
  • 4
    These are just HEAD requests - they should be some of the smallest http requests you can get. What kind of bandwidth use are you seeing to make you worry about overall bandwidth use? – Xiong Chiamiov Dec 26 '16 at 17:23
  • 4
    Also, if they're getting 404s, that doesn't really matter - it means they aren't successfully finding vulnerabilities. Any filtering you put in place has a decent chance to hurt legitimate traffic, so unless this is actually causing harm to your site, it's best to ignore it. – Xiong Chiamiov Dec 26 '16 at 17:32
  • 2
    You can certainly write a custom fail2ban match for this sort of thing. – Michael Hampton Dec 26 '16 at 19:08
17

You could drop packets from tor nodes all together if you like with ip tables. List of tor nodes can be found at:

Ref 1: https://check.torproject.org/cgi-bin/TorBulkExitList.py
Ref 2: https://www.dan.me.uk/tornodes

Here is a bash one liner to block all traffic from tor to your web server. There are currently about 2000 tor nodes online now that support port 80. So you will be inserting quite a few iptable's wouldnt really call this a long term fix but should stop the attack.

wget "https://check.torproject.org/cgi-bin/TorBulkExitList.py?ip=8.8.8.8&port=" -O /tmp/ip.lst && sed -i -e '1,3d' /tmp/ip.lst && for i in $(cat /tmp/ip.lst); do iptables -A INPUT -p tcp -s $i --match multiport --dports 80,443 -j DROP; done
  • Did this and that seems to work. Iptables is piling up with Tor servers now which is a good thing. – Alex Dec 26 '16 at 7:51
  • Hmm, it started again...guess not all IP are on that list :( – Alex Dec 26 '16 at 7:54
  • 9
    Or your attacker has switched onto a different proxying network. – Xiong Chiamiov Dec 26 '16 at 17:21
6

You can drop packets that contain specific string and apperantely all requests contain: 'HEAD //images/' string.

I suggest the following rule for now and later you can remove it:

iptables -I INPUT -p tcp --dport 80 -m string --algo bm --string 'HEAD //images/' -j DROP

  • 1
    You could do this, but they just move to different folders, and you will still end up with 100's of rules. – cybernard Dec 26 '16 at 4:42
  • If this was the case, the user can block "HEAD " request in the same way and in 99% this will not affect the functionality of the website. – Opaida Dec 26 '16 at 5:17
  • Hi, i tried this, but it didn't do much. I even tried it with only HEAD, but it didn't stopped any request. – Alex Dec 26 '16 at 7:38
  • Make sure about the HTTP port.. the rule considers you are using port 80 (default). – Opaida Dec 26 '16 at 7:51
  • 6
    @Alex It looks OK from here but the best way to ensure this rule is well written is to disable the TOR-related rule you wrote before. Especially knowing that some people can legitimately be using TOR while using your website properly - you don't want to ban these ones. – Shlublu Dec 26 '16 at 8:56
3

If your security model permits passing traffic to third party, then the easiest and most effective solution may be to front your application with Cloudflare. At the time of this writing, Cloudflare pretty much automatically blocks all Tor connection unless they solve a CAPTCHA page and they also have Web Application Firewall you can configure to further filter traffic before reaching your server so the attacker don't consume your server bandwidth.

Otherwise, you may have some luck by configuring fail2ban. fail2ban is a piece of software that puts temporary IP block by changing the OS firewall policy when someone fails authentication too many times. In your case, fail authentication would be someone that keeps 404-ing.

  • 4
    atm i can't use Cloudflare because i use a EV SSL. I need to have a business account from them and my business isn't that big yet to pay 200USD per month just for the website. – Alex Dec 26 '16 at 10:07
3

My web server has this all the time, and after months of examining logs I can see there are 100's of different things they will eventually ask for.

One of thousands of bot networks is mapping your apache for vulnerable components. Hoping to get lucky, with a common component that it knows how to exploit. I get bots searching for phpmyadmin all the time. Along with dozens of other things.

Then of course if does get lucky, they will eventually deploy further bad things so they can use your computer for their own nefarious deeds.

Virtualize with something you can install a modern kernel, so you can use ipset. Maybe KVM, docker uses the built-in kernel so that is no good.

There botnet is likely 10's of thousands of computers so you won't be able to effectively block this, especially not without ipset.

Also as long as you are using TOR they have access to even more IP addresses from random people.

Option #2

Replace the HTTP_NOT_FOUND.html.var file found, at least on my system, /usr/share/apache2/error

You could create a HTTP_NOT_FOUND.html.var.php file then change the apache file,/etc/apache2/errors.conf, to point to said file.

Then the 1st attempt would be the last.

  • All i found out is that it is a scriptkiddie which found a script to search for something through the TOR network. I'm glad it is not a botnet :) – Alex Dec 26 '16 at 8:17
0

First, don't panic and make sure you have a towel with you!

From the information you have provided, I don't think you're being attacked, just scanned. These days, this is 'normal' behaviour and something you're likely to see on a regular basis. Think of it as someone casing your joint, looking for an open window or unlocked door. They are just looking to see if there is an easy way which they can use to gain an entry.

There are many scripts and tools out there, as botnets which just scan the internet looking for possibly vulnerable systems. Whether this is a prelude to something more serious is almost impossible to determine (yet). A lot will depend on whether your site looks valuable and to what extent your system is up-to-date.

  • Make sure your software is all patched and up-to-date
  • Make sure you have disabled any services running on your server which you don't need/use
  • Make sure your web server configuration is correct i.e. only has enabled features you use, has standard things, like a correctly configured robots.txt, does not have enabled any modules you don't need etc.

You mention this traffic is killing your bandwidth. This makes me suspect you're running the web server on a system with a low bandwidth as these requests are quite light weight, i.e. just HEAD requests. If this is the case, perhaps you need to re-think whether running the server is actually a good idea.

When your putting a server on the Internet, you're putting up something which is public - you're saying "Hey, I've got a server here, come and have a look". You need to be prepared for when someone will come, and you have no control over who does. Trying to block this sort of traffic will send you mad - if you let it, you will just end up in a game of 'wak-a-mole'. Don't even bother trying to block this level of traffic because as soon as you do, you will likely just get something vary similar from a different IP address. You will end up spinning your wheels writing filters and adjusting firewalls.

Setting up a server and putting it on the web is almost a trivial task these days. However administering such systems is not, and something you need to consider carefully. Putting it on a server at the end of a home or small business DSL link or similar is almost always a bad idea. It will likely have a hit on your bandwidth, will frequently take more time to administer than you expect and you're unlikely to have the underlying infrastructure necessary to provide a robust and reliable service.

My suggestion would be to consider hosting your web service with one of the many different hosting services out there. There is a large variety of choices and you can almost certainly find something which meets your needs from low level Infrastructure as a Services (IaaS), through to Platform as a Servide (PaaS) and Software as a Service (SaaS). Each service offers varying levels of flexibility and varying costs which you can select from. You will be able to find something with the right balance of cost and features with enough research.

If you don't want to pay to host your service, you really need to re-consider what you're doing. There are costs associated with all forms of hosting. The difference is how those costs are absorbed - either you pay with money or you pay with time and your own resources.

  • First, i am not panicking ;) second, i have enough bandwidth (30TB per server), but that doesn't mean that someone just can eat it because they like it to conquer a server. I am just monitoring my server very closely. The reason why i wanted to stop this "attack" is because it is a mail service and they were trying to hack into it to abuse my service to spread spam. So, for me personally i wanted to stop this attack right away to let them know that it is worthless to try. I am a server admin with 20+ servers online, so i know what i am doing and i just wanted to stop this. Happy holidays. – Alex Dec 30 '16 at 22:23
  • Not sure what you mean by 30TB per server with respect to bandwidth - sounds like storage size rather than network bandwidth. Regardless, you would need to be seeing a LOT of these HEAD requests before you would see any noticeable impact on your bandwidth assuming a reasonable connection i.e. !Gbps or better. At any rate, this sort of scanning is vary frequent and while it is good to monitor, preventing it is near impossible. Assume you are always under 'attack'. A focus on hardening rather than blocking will generally give a more robust and effective defence. – Tim X Dec 31 '16 at 0:40
  • No it is bandwidth. Like i said, i know what i am doing here. I am a Red Hat server admin on a highschool for several years now and have for my private company already 20+ servers running in a DC. I know what bandwidth is and what hdd space is lol. That said, at the moment of the, according to you, innocent attack my server got abused by someone who made several accounts and was spamming around the 10000 mails per day while i have rules like one recipient per 30 seconds. If at that time also a scanning job is running, then i can only acknowledge that i am under attack and need to do something – Alex Jan 1 '17 at 1:50
  • I'm not questioning your knowledge or experience, I'm saying I don't understand what '30Tb per server' means with respect to bandwidth. I'm accustomed to bandwidth referring to amount of data transferred over an amount of time. For example, where I work we have 2 dual core 10 Gbit per second links. We have a class B network with a few hundred servers and about 4k desktops. Your metric has no reference to time, so I don't understand what it means wrt bandwidth. I do know researchers managed to achieve 1Tb per second over optic fibre for the 1st time in 2016. – Tim X Jan 1 '17 at 22:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.