0

I'm thinking of rate limiting anonymous requests based on IP, but for users with a valid authentication token, I can't decide if it's better to log and rate limit requests based on their UserId or IP address.

Option 1 : 15 requests per minute per UserId

Option 2 : 30 requests per minute per IP address

Option 3 : Do both

If a malicious person/entity decides to run a bot attack against the APIs, the best option seems to be rate-limiting them based on IP address because they can potentially have a lot of tokens ready for users with different, say, email addresses (assuming each user registers with a unique email address).

However, there is also the possibility that a sophisticated individual can switch IP addresses during the attack but pass the same authentication token and abuse the system with the same UserId.

The third option seems enticing, but for me, it means every API request needs to be written twice to the database (records are horizontally partitioned based on UserId and there is another partitioning scheme based on IP address. Sharding will make it easier to look up and count requests).

Are there any best practices as far as which option (or something else) to pursue when rate limiting authenticated users? I'd like to have a general idea of what to do before I'm deep into writing all the validation logic?

  • You can do both. For example, a login endpoint you can limit the requests and also limit the number of request per email or username, or any unique identifier that you use. – David Magalhães Nov 13 '18 at 22:01
  • There is no need for a (SQL-like) database to do rate limiting. You can use an in-memory key value store (possibly as simple as a plain hash table with LRU eviction) with one counter and one timestamp per IP or user-id. You don't need to maintain a complete access log to do rate limiting. – Future Security Nov 13 '18 at 22:12
  • @FutureSecurity I'll be using Azure's Cosmos DB, and I remember exchanging emails with one of their team members, and they recommended that Cosmos DB can make an extra caching layer like Redis unnecessary as it's designed to be fast already. Also, a complete access log can help me run different kinds of analysis on APIs until the log from past dates is flushed out so it doesn't get humongous. – user246392 Nov 13 '18 at 22:23
1

I would say it depends. In generall IP blocking is a good practice because the IP is the only identifier that exists for anonymous users. But keep in mind that a user can "switch" IPs using online proxy services AND on the other hand that maybe multiple users share an identical IP when they use a proxy server (which ist often the case in enterprise environments).

If your users are authenticated then you should block the userId when the user is doing malicious stuff.

Another aspect, how easy is it for an anonymous user to create a new account in your app? When he can create new accounts without a problem, then blocking userIds is useless, because when the attackers user 1 ist blocked, He easily can create a new user.

So you should rely on both techniqus and check how easy it is for an individual to create new accounts.

  • Totally agree with the fact that using IP uniqueness isn't really a good idea .... – binarym Jan 2 at 17:17
0

I will add the use of geoip, in some cases is desirable that you can block directly countries that don't have nothing to belong to your service as well as white listing for the countries that you have more trust.

0

Both. They protect against different styles of attack.

Instead of sharding the database, you could look at using Fail2Ban for IP blocking and stick with application/internal monitoring only for user blocking.

As long as you have a log somewhere for requests, you can grant Fail2Ban read access to do its job without even touching your database.

If this is your application, you can implement a minimalist log just for Fail2Ban. If it's a commercial application, then you'll have to find out how and where they log inbound requests.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.