Is there a problem with this approach to handling invalid web probes?

Question

Like any other website owner, I get frequent probes for vulnerabilities e.g. .php .sql or .gz pages.

These used to appear in my log files as 404 responses (we host on ASP.NET Core). These also take up server time and processing as it has to

1. handle the request in the pipeline
2. check the static files for a match
3. check the routing table for a match
4. redirecting to the error handler middleware
5. log the error
6. show the 404 error page

Using postman I see that a generic 404 returns about 5.44kb in HTML

So I've added a middleware handler for requests that checks for requests ending in suspect extensions such as these, which just clears the response with no further processing. This response is now 129 bytes but returns a 200.

My question therefore - Is this a safe approach? I can change the response to 404, although I doubt they'd think the blank response a valid one.

Answer 1

You don't need to care about if the status you return is the most appropriate. It's a scan anyway, and if your code returns a blank page with HTTP 200 status and the scanner segfaults and catches fire, it's not your problem. You must care about legitimate requests, not much about non-legitimate ones.

On my lab servers (online with public IP) I use fail2ban: 4 HTTP 4xx errors bans the IP for an hour. On some places I use a tarpit just for fun, such as LaBrea PHP Tarpit.

Answer 2

If you are using a Windows server you'll have to find suitable software for the job but here is an idea.

I manage a number of Linux servers and Fail2Ban is often used to thwart brute-force attacks (SSH/POP/SMTP etc). There are alternatives (I personally prefer CSF-LFD). These tools act by watching the log files continuously and block IP addresses at firewall level (iptables or other). This is very effective and relieves your server that would otherwise have to handle hundreds of bad requests for nothing.

What I have done is add custom rules so that if an IP address generates too many 401/403/404 HTTP errors over a certain period of time, then I block it.

Of course there are solutions, you can have fun with hackers script kiddies and frustrate their efforts by returning bogus statuses. But blocking them is easier.

I don't know what kind of Fail2Ban-like solutions are available for Windows, you have to research the market. But I think this should be the job of an IDS, there should be something more comprehensive in place than just blocking these specific 'attacks'.