How to effectively test for username enumeration risk?

Question

I am responsible for some public-facing web applications, and I want to test these for the username enumeration risk as explained in Testing for User Enumeration and Guessable User Account (OWASP-AT-002) by the OWASP project.

These applications are .NET based websites, with a standard login consisting of username/password and typically a password reset functionality.

I could do this by hand, of course, (at least for visual differences) but I would rather use an automated method.

The test should include:

• (Visual) Content-Based differentiation (simple for a human tester)
• Response (HTML-Text) based differentiation
• Time-Based differentiation (Response faster/slower upon match)
• others I forgot?

The test should not include (real) password guessing, just username enumeration. I have a set of some known user names to test with, along with the login screen URLs.

What is a simple and effective method to test for a username enumeration risk?

Answer 1

visual differentiation is a consequence of HTML differentiation. It cannot happen that the same HTML + JS + CSS will lead to different visualizations (barring some intentional fudging like, I don't know... time-dependent background colors or animations?).

So you can send out a sufficient number of queries for real users and compare the same queries for fake users; and you must do the same with all unauthenticated endpoints that are accessible.

This means examining not only the "login" URL but also the "I've lost my password, how do I go to recover it?".

For each such endpoint, in addition to the differences in outputs, calculate the average time to answer and its variance (a high enough variance will negate any reasonable timing attack).

You may want to do this twice, i.e. you check user "marcel" once and then again. The series of the "firsts" we call the unprimed series, the series of the "seconds" is the primed series.

Purge the received contents of any timestamp, unique id or user-related information (e.g. "Sorry, password for user marcel is incorrect" should become "Sorry, password for user USERNAME is incorrect" - simple regex here. Careful if some user is called table, html, or body, since that would skew the results).

Then the desired outcome is that, for all endpoints (1),

1. all the answers for real users and fake users are identical.
2. the removed parts ("marcel", "admin", "20160920173511.128", etc.) are either all identical, or all different. It mustn't happen that real users get a real timestamp and fake users get no timestamp or a differently formatted timestamp, for example.
3. the time to answer for each fake user should be very close to the time to answer for a real user. Ideally you want the same mean and variance.
4. the times distribution for the primed and unprimed series of real users might well be different (usually, the primed times are shorter). If they are, then the same ratio must hold between the primed and unprimed series for fake users.

Point 3 means that looking for a user that is not in the database must take the same time as looking for a user that is in the database and has data to be loaded and analyzed. Achieving it isn't trivial. I've seen random delays added to the process, but they must be based on the user name (same user name = always same delay) (2).

Point 4 could lead to a twist in timing attacks, allowing to distinguish between real and fake users even if for a single attempt the reaction times are the same.

(1) I've seen several systems that will ask your username to help you recover the password, and cheerfully say "Sorry, that username is not known!". A more suitable message would be: "Very well, lserni (if that's really your account), if you're in our database, you should receive an email shortly."

(2) When a username comes by, I want to add a random delay that will thwart most attempts to time the authentication process. marcel is resolved in 85 ms and juniper in 62 ms. Is that because marcel exists and juniper does not, or because their delays were different? You must not know. But what happens if the delay for user juniper is random every time? It happens that I can average resolving time, and the randomness will slowly (depending on its variance) cancel out. So I add to user juniper a delay which was randomly assigned to juniper, but is always the same whenever juniper tries to authenticate. To avoid saving things in a huuuuuge database, what I really do is calculate a 32 bit CRC from the name, XOR with a secret salt, use the result to seed a PRNG, and extract a random number. At that point even if the attacker knows that existing usernames get a 50 ms delay while unknown usernames get 30, he/she cannot know whether juniper's 62ms were given by 50 plus a random 12 (juniper exists), or by 30 plus a random 32 (no juniper). And the 12 (or 32) will not be the same for a different user who will have a different CRC32. So using the same username will not help, but using a different username will not help either. Of course, I need a random range large enough to "overwhelm" the known/unknown time difference.