Should I block visitors who seem to be probing my site?

Question

I have detected some attempts to probe the security of my site. Should I blacklist their IP address? What are the considerations or tradeoffs to use in deciding when to block their IP address and when not to?

Answer 1

As a rule, the returns on this type of defense are nearly zero. There are exceptions, but even then this technique may offer only a small amount of security.

Exhaustive vulnerability scans of randomly-chosen hosts offer extremely low returns on scanning resources. Instead, successful attackers generally choose one of the following two options to increase their odds:

1. Check for a very small number of vulnerabilities over a large number of hosts
   This includes bad passwords for common services such as SSH, POP3, cPanel, Wordpress, Joomla, and the like. If your root password is "root" or your site admin password is "admin", you can expect to get caught in one of these drift nets surprisingly quickly.

2. Starting with a single known vulnerability, use a search technique to narrow down your target list to only the most likely victims.
   Usually this starts on one of the common vulnerability databases. He finds a recent candidate and identifies some string that would be common on vulnerable hosts. This is often a "Powered by" line or a login link or a specific URL pattern. Then, using his favorite search engine he retrieves a list of sites that follow that pattern.

The important point is that in neither of these cases does the attacker make more than a few attempts at a given site. Either they're successful immediately, or they move on.

That's not to say that you won't see the occasional persistent attacker. There's always some misguidedly optimistic individual running Nessus against a /16, but such attacks are by their very nature self-limiting and very rarely successful, since the vast majority of the attacks target software that a given server doesn't even run.

Could such an attack be successful? Yes. Does it ever happen? I deal on the average with several compromised servers per day, and so far I have yet to see even one come from nessus-style full-spectrum scans. But the probability isn't exactly zero. And in the world of random attacks, statistics and probability matter quite a great deal.

The exception is when the attack is not random. If you run a high-value or high-visibility site, you'll attract a lot of direct attacks against you in particular. Sites like Facebook, Twitter, Hotmail, etc., see an extraordinary amount of attack traffic. If you deal in this type of site, you probably already take security very seriously and aren't looking for advice.

A note about SQL injection: This type of attack does take some refinement; after catching an SQL error message in a cursory scan, an attacker may send several dozen queries to a given server to get the bits centered right. So theoretically a clamp-down response may prevent such an attack... but probably not, since he'll probably just return using a different IP. Since he's already alerted on your server, he won't be deterred by a blacklist. Better advice is don't run crappy software. There's no excuse for using non-parameterized queries today.

Answer 2

Arguments in favor of blacklisting: Might as well make the attacker's life harder. While this is not a strong defense, it might give a little bit of additional time to investigate the attack and put in place stronger defenses.

Arguments against blacklisting: In the case dynamic IP addresses, blacklisting them may end up blocking other benign users. You might never know that you are blocking other benign users, or it might lead to hard-to-diagnose errors in the future. In addition, there is some potential configuration complexity to managing a blacklist.

If the attacker is attempting to disrupt the site by overloading it (to mount a denial-of-service attack), then blocking the IP address might be the right answer. A temporary block might be enough.

If the attacker is just probing or scanning, I would be less inclined to blacklist their IP address, but you can take the tradeoffs into account in each situation and make an informed decision about what makes the most sense.

Answer 3

It obviously depends on what your site does, the downside of blocking IPs (could you inadvertantly block real non-attacking customers), and the risk that something could be compromised (is it likely that some users have weak guessable passwords on the site?)

Short term blocks seem perfectly reasonable varying from anywhere from 5 minutes to a day. This may allow an attacker to try ~1000 attempts a day, versus a ~1000 per second in an online attack. Granted, you need to be sure that you do not block legitimate users who forgot their password.

Personally, for example, I block large ranges of IP addresses from China/Russia on a website I run for a local business. I observed that the ssh server for the website had a couple thousand attempted logins with non-existent usernames. While they weren't close to guessing my username, let alone my random high-entropy passphrase, I wasn't comfortable with the attempts and so no reason to let them continue. So I installed fail2ban with loose settings (five failed logins = 30 minutes ban; whitelist for my static IPs -- so I can still login if needed -- also I login 95% of the time with a RSA key so a single failed login is rare). I also changed the ssh port from 22, yes this is only security by obscurity that's easily defeated by a port scan (though I could setup a honeypot for that; but didn't), but that's ok if its part of defense-in-depth. Implementing these changes caused the number of attacks to go down to zero in the past month (as far back as the logs go).

Answer 4

Edit: This answer does not consider login brute-forcing, just attacks which attempt to exploit system vulnerabilities. Of course it seems legit to me to temporarily ban an IP which attempts 30 logins in 5 minutes. /edit.

If you don't have an automatic system in place to check and block requests, don't ban IP addresses that attempt to hack your service. If you block them, they can't make anymore requests, and you won't know if there is a leak. Attacks are going on day and night, and more often than not, you won't be watching the log while an attack is attempted. Block an attack now, and some day someone else will find the leak.

Of course, blocking attacking IPs is a precautionary measure which will help your security. It is not real security, but it might just ward of an attack while giving you early warning.

Doing this in an automated manner might have an advantage when you give a default response back (like a HTTP-200 response) so that automated tools won't detect the block. If you keep logging the requests, you can later check if any of the attempts would have been successful.

There are some problems with this though:

• Attacks may come from a botnet, hacked server, or dynamic IP. Sooner or later, you're going to block a benign user. At least just temporarily ban the IP until you've made sure that the attack would not have been successful.
• It is a lot of work to check all requests from detected attacks, especially since there are so many automated attack tools that make hundreds of requests.
• You will probably start to rely on the detection system in some way. This kind of precaution will make you feel safer, and thus make you think less about security while configuring the system (or writing code).
• You still have to check log files from time to time to make sure your detection system didn't miss anything.
• False positives may occur. A simple example is when detecting apostrophes in POST-data (which an attack tool might attempt to detect SQL-injections), you could block a user who is trying to use an apostrophe in their password. This may be solved with a captcha, but targeted attacks will also solve the captcha.

Overall, I would not block any attempted attacks. There may be situations where this is a good idea (perhaps in high-security environments, like for a bank or website with millions of visitors), but in general I don't think it will really help, or may even give you a false sense of security.

The only exception is a DoS attack. Here is no security risk, the only thing being exploited is processing time or bandwidth, so there is no reason not to block this. Though even then, only temporarily ratelimit or ban: these attacks are likely done from a botnet, thus making you block user's IPs.

Answer 5

I would monitor the situation a bit more before moving on with blacklisting an IP, even temporarily. Besides the cons mentioned by D.W., I don't think a serious attempt at attacking your site would employ the same IP more than a couple of times; it's not very hard to spoof it after all. If, however, after a short time spent monitoring, I were to observe repeated probes from that specific IP, I would definitely consider blocking it.

Another thing I might be in the lookout for is the nature of the probes. If the range of offending IPs all share common characteristics in a given window of time, there is a good chance they have been made by the same attacker.

Finally, if the attacks were persisting and / or I had some time to spare, I would attempt to get an idea of what the attacker sees by repeating the probes myself. At the very least, I would get more information about my site 's security.

Answer 6

This is an "it depends" tradeoff.

1 - what is the priority of accessibility vs. obstruction of attackers?

For any block, you want to determine whether you are blocking legitimate usage as well as attackers. For example, if the IP comes from your customer and one of their machines has been hacked to scan you - you may want to alert the customer and work with their security team rather than shooting off your own foot.

2 - what is the impact?

Do you have the helpful protocols that expose your network turned off? What can the attacker see when they port scan you? Scanning yourself on a regular basis is a good practice, so you can know what you're vulnerable to. If you've locked things down fairly well, your risk here is minor.

Also - what damage is the scan doing? Is it slowing down legitimate usage?

3 - what is the nature of the risk? What is your capability to manage it?

The typical classics - availability, compromise of assets or information, reputation of the organization. What can you loose from the port scan, what can you loose if they find something and how frequently is it occurring?

And what is your staffing and your capability to address it? In some networks, putting in a block is fast and easy, in others it takes a significant degree of due diligence. If you put it in incorrectly, what is the harm?

Answer 7

The most reasonable thing right now is to follow that scan and make sure that attacker didn't find anything vulnerable in your application. Secondly you should check that IP. Blocking someone's IP address isn't always the best solution. The attacker could be hidden behind the NAT. So if he shares his IP with a lot of different users, you will lose visitors (they will be also blocked). The scan could be also started from the innocent victim (his PC could be a part of the botnet). However you should make sure that:

• this IP doesn't belong to any bots/web-crawlers (which are scanning the Web almost all the time)
• the traffic generates by this scanning doesn't generate huge bandwith (or doesn't affect the performance of your server).

Answer 8

I suggest a different approach to the problem rather than banning ip addresses or fail2ban, which can be fragile and ineffective. Instead I recommend changing the configuration of your webserver so all request to unknown server names or locations return 404. I wrote about this technique for nginx here:

https://medium.com/quiq-blog/harden-nginx-from-malicious-scanners-5bd1ca9c5a5c

Answer 9

If you have annoying visitors, site scrapers, or spammers, you may find it useful to block these users from accessing your website content. You can block bad visitors by IP address or blocks of IP addresses using a .htaccess file.

Read more some useful examples: http://kb.mediatemple.net/questions/1699/Block+a+specific+IP+address+from+accessing+your+website#gs