For the past few weeks, I am observing a lots of visits from a specific user-agent - Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.135 Safari/537.36 to one particular URL.

The user-agent corresponds to chrome 84 on windows 10 desktop. This browser is an old browser and the overall contribution to traffic is lesser than 1%. But for the particular URL I am speaking about, 95% of the traffic is from this user-agent. Bounce rate is 100% for these visits. What is more confusing is that all the visits are from unique IPs and I couldn't find a pattern (like common location, revers DNS host, etc.,) in them.

I am clueless about where to start and how to proceed. Could this be a bot activity? If so, is there a way to find and prevent using firewalls like mod_security.

P.S: I am using apache 2.4 server with mod_security and OWASP ruleset 3.3.

  • 3
    I am clueless about where to start and how to proceed. Why are these requests a problem? When you run a public-facing web server, it will receive all kinds of bogus requests. That's normal. Trying to block these requests (or other illegitimate requests) is futile. Just make sure that your server does not have a vulnerability that could be exploited by these requests (or others) and go on with your life. Or, if you're really concerned, put your web server behind a WAF like Cloudflare.
    – mti2935
    May 25 '21 at 16:57
  • OP, can you post a few of the IP addresses that these requests originated from?
    – mti2935
    May 25 '21 at 18:56
  • 1
    It sounds like whoever/whatever is doing this may be trying to scrape your site to collect the data that your site provides. They might be proxying their requests through a bot net so that they come from many different IP's instead of one. Placing your site behind Cloudflare's web application firewall (WAF) will block most of these requests from reaching your server. See cloudflare.com/waf
    – mti2935
    May 26 '21 at 10:15
  • 1
    Thanks for providing more clarity on the problem, and why you feel that it's important to solve it. These types of problems often turn into a cat-and-mouse game. You could look for something unique to use as a condition to block these requests (such as the user agent), but they'll pickup on what you are doing, and change what they are doing, and it will just go on. Cloudflare uses AI and ML to detect invalid requests and block them. It's much more advanced than anything you can do with mod_security. Their system is designed to play this cat-and-mouse game, and win.
    – mti2935
    May 26 '21 at 13:02
  • 1
    Also, I checked the IPs that you posted to see if these are TOR exit nodes, or if these are on any lists of known abuse IP's, bot nets, etc. If so, then maybe a solution to your problem could have been blacklisting IP's based on these lists. But, these IPs are not on any of the lists that I checked.
    – mti2935
    May 26 '21 at 13:06

Based on conversation with OP in the comments following the question, Cloudflare's web application firewall (WAF) may be a solution. See comments for more information as to why OP feels that this problem needs to be addressed, why this problem may be difficult for him to solve on his own using mod_security, and how Cloudflare's WAF may be a more effective solution.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.