I have been receiving a great many emails recently reporting issues within my website that, when gone back and looked up, have been reported as being Brute force attacks. Upon further inspection of their location. I have noticed a great deal of them are coming from the same IP address, and seem to be linked to Google, or Yahoo for example this site:


reports the following IP:, Which originates from google cloud, as brute force.

Is there a good way to distinguish between apparent brute force attacks, and Crawlers checking the site?

Update: all emails are being generated from production site. The error reporting is issuing a null reference exception.


Brute forcing usually applies to sending user/password combinations to your login endpoint. So what you see in a brute force attack is thousands of HTTP POST requests to the URL behind which your login logic resides.

Crawling has a completely different profile. A crawler will hit your website at any entry point, most likely the site's homepage, and start following links from there. Benign crawlers never send POST requests; they simply GET each page, look at the links on that page and later on GET the linked pages. Well-written crawlers will also honor robots.txt, so if you list your login endpoint in robots.txt, crawlers shouldn't request it. Plus well-written crawlers follow rate limits you suggest.

The source IP is completely irelevant in determining whether you're dealing with a crawler or a brute force attack. Both crawlers and brute force attacks may originate from a single IP, or be distributed over several or even hundreds of different IPs.

  • Thank you very much for your response. I didn't want to start forcing a block if it wasn't necessary, especially if it was going to affect potential traffic. – Joshua Mar 1 '17 at 17:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.