What criteria should be used for manually reviewing / prioritizing failed logins?

Question

I have a web application that is exposed to the Internet, and am interested in creating alerts, or prioritizing the manual review of failed logins to the web application.

• What thresholds, or formulas should be used to determine when a failed login should be reviewed in more detail? (Assume that an alert could be triggered on a per-user basis, or a per-IP basis.)

Some formulas I tried, but had issues with consistently applying include:

• Success / Failed Logins Per IP
• Success / Failed Logins Per User
• Has the user ever logged in successfully at this IP before? (if not, be more conservative)

I ran a Proof of Concept audit against existing web applications and noticed that there were a few legitimate use cases for the app that skew the straightforward application of the above ratios:

1. The user is behind a shared NAT or Proxy (remote office).
2. The user is on a home device (or mobile) and the source IP is always changing.
3. The machine is shared among many users, and that has a constant IP but many users

Some things I'm trying to prevent (incomplete list) include:

1. Brute forcing usernames or passwords. (the lockout period is 24 hours)
2. DoSing the application by intentionally locking out users
3. Any attack above from a BotNet or TOR
4. Internal attacks / suspicious activity from internal users.

The ideal solution would have a consideration for the scenario below, but any general approach, academic whitepaper, or research would be helpful.

• Optional Scenario:: An attacker is using sock puppet accounts to add success logins to the failed account, making him appear as a NAT above.

Answer 1

At face value, this is easy, but as anyone can tell by reading your question (or for those of us who have implemented something along these lines) this quickly becomes a can of worms.

Lets look at a few potential metrics...

Failed attempts per username -- the standard metric by which an account gets locked out, keep it low and most attacks will be blocked.

invalid usernames per IP per x hours -- Seeing 100 login attemps for unique invalid usernames in an hour from the same IP is a good indicator of malicious activity.

locked out users per offending ip over last x hours -- If one IP has caused n number of users to get locked out in the last x hours, that's suspicious. You'd want to compare this to the number of successful unique users logged in from that IP. If there has been one good login and 40 locked out users, something fishy is going on and worth banning the IP until an investigation can confirm.

avg & max valid user logins per IP over x hours/days -- gives you a good baseline for what standard use might be from NAT'd networks.

The key is to build baselines and flag any anomalies either for manual review or automatic action. That automatic action could be locking out a user, banning or rate limiting an IP, flagging an IP or user as suspicious and requiring all logins from that source to go through additional bot prevention steps i.e. CAPTCHAs and or email verification.

Some sites require these additional verification steps from any user logging in from a new device, this too will greatly decrease the potential for a successful user based attack.

As for your optional scenario, lets say they are using sock puppet accounts to meet baselines to appear as a NAT environment. Lets say they have 100 accts logging in successfully. If they are trying to get into a specific account, that account will be locked out after 3-4 tries. Assuming you have a decent password policy, attack prevented. Lets say that they're trying to get into one of 10 accounts, would 10 accounts being locked out in an hour from a NAT environment be normal? Doubt it.... Flag the IP for suspicious activity requiring users to verify their humanity after one failed password attempt.

Ensuring that users get locked out in a reasonably quick time will prevent most brute force attacks. Adding IP based metrics for locked out users, invalid users, and other related metrics compared to baselines and tied in with standard bot prevention and user verification methods will further decrease your risk and ensure happy auditors, sysadmins and users.

Answer 2

Difficult for a straight up answer since I have no indication of how many users you have, or intend on having, logging into your server. If this hits the thousands, you will be shooting yourself in the foot with so many false positives, that you will eventually ignore all alerts.

So I will add my two cents to this devils advocate style:

• Brute forcing usernames or passwords. (the lockout period is 24 hours)

On if someone brute forcers a legitimate user name, they'll trigger an alert blocking a legitimate user. Multiply this by N and you now have N amount of irked users.

• DoSing the application by intentionally locking out users

This is also "unintentional" see above"

• Any attack above from a BotNet or TOR

This is also "unintentional" see above"

• Internal attacks / suspicious activity from internal users.

If you have to worry about internal users, you have bigger problems.

Here is what I would do, and have done: Because most of my userbase that needs to "log in" to anything, typically come from defined places (an ISP, a business, etc.) I tend to allow those in (via firewall) and block the rest out. This is general practice however, its not for everyone, My method would be the following:

1) Captcha or Cloudflare - the reasoning for this is simple; most "attacks" nowadays are mainly automated. Captcha and Cloudflare will minimize those types of attacks.

2) Strong firewalling / ACLs - if say you're in North America, do you REALLY need your server visible to APNIC, RIPE, LACNIC, AFRINIC networks? (Google those if you don't understand what they are)

3) SIEM logging and monitoring - again, tough thing to even mention since I don't know the size of your network, logs, etc, me though, I baseline normal traffic for a quarter, then ignore ALL KNOWNS, then alert for unknowns before blocking.

Networks differ, so you'd likely yield a better answer with better description. E.g., I have a web app, that is accessed by N amount of users. I have/don't have a budget, etc.