0

My public website is getting a lot of get requests to endpoints like "wp-login.php", ".env", ".git/HEAD" and many others. This seems to be a non-trivial portion of my website's traffic and I'd like to discourage it or at least reduce its impact.

Option 1.

I've heard of tactics such as redirecting such requests to random 10 hour long youtube videos, like so: https://twitter.com/nick_craver/status/720062942960623616 Is this effective against a significant portion of these actors?

Option 2.

I could block ips that make requests to such endpoints by using something like this:

var http = require('http');
http.createServer(function (req, res)
{
    var ip = req.ip || req.connection.remoteAddress || req.socket.remoteAddress || req.connection.socket.remoteAddress;
    if (ip == '127.0.0.1') // exit if it's a particular ip
        res.end();
}

source: https://stackoverflow.com/questions/12349251/restrict-access-to-node-js-based-http-server-by-ip-address

or something like this: https://www.npmjs.com/package/express-ipfilter (though this seems to require a redeployment to add new ips to)

Option 3.

My current plan is not especially elegant, but the idea is to just never respond to a request to any of the known get requests I'm receiving, not sending them a redirect to youtube:

const bannedGetRequests = ["latest/dynamic/instance-identity/document","wp-login.php" ...]; <-about 1,500 that I've got in my logs now.

bannedGetRequests.forEach(request => {
    app.get(request, function (req, res) {
        return; //I'm not calling res.end() here on purpose, I'm hoping this will leave their connection open awaiting a response I'm never going to send, if this slows them down even a few milliseconds, it seems worthwhile to me.
    });
});

Option 4.

Pay cloudflare. Their free tier does not seem to support bot suppression. Maybe it's not as big a deal as I think it is. I don't really want to go through what looks like a huge amount of work to setup some kind of support from Cloudflare only to find out it doesn't even address my concern. Furthermore, this is a web application, I don't really think that using a CDN is going to be especially helpful to me. I don't have very many users (aside from these bots I guess).

Is there anything I'm missing? Are any of these options objectively superior/inferior to others?

Additional information: This is running on an AWS elastic beanstalk which puts an Nginx server (from AWS) in front of it that I'm not sure I have any control over.

1

There are likely many tools and custom scripts out there that accomplish what you are trying to do. However, I think fail2ban is a good example. It ships with profiles "apache-badbots", "apache-noscript", and others that that will follow your logs and temporarily ban anyone who is in violation of the rules, e.g. trying to access .php pages on a NodeJS server. These profiles can be modified to suit your configuration.

That said, this type of activity is very normal on the internet, and there is likely very little risk in leaving it be.

| improve this answer | |
  • I don't think there's much risk, but it can drain the available bandwidth. I too have great success with Fail2Ban. There will be a bit of a learning curve for setting it up, but afterwards it should significantly diminish the offending traffic. – Vilx- May 13 at 22:52
0

Cloudflare's Attack Mode does offer bot suppression for free, however the screening it offers is very aggressive (5 second wait before going to the site, captcha, etc).

You should configure your site to run "silently" - configure it to not respond to pings, close any ports you aren't using, and timeout any request that doesn't lead directly to a valid page.

But honestly, I don't see why you're concerned. Unless this traffic is doing damage, it seems like you're putting a lot of effort into something that can't hurt your site. These vulnerability scrapers will always exist, and there isn't a cheap & easy way to screen them out. If you're concerned about Denial of Service, and your site can't hold up to the "ambient noise" of bots, you might consider upgrading your infrastructure.

| improve this answer | |
  • I would like to agree with the "why are you concerned?" and it's not because I don't understand why you would like to stop all those requests. It's just that I know from my own experience that you will end up spending more time preventing these requests and monitoring the logs then you can afford. It is imho better to invest the same time in making sure your site is secure (latest patches and updates etc). That said, I encourage anyone to try to find a solution to the problem with these kinds of requests, it would be nice to have! – Don King Apr 12 at 12:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.