What are some potential attacks that could be done to a website that doesn't manage url-encoding properly?

Question

A month or so ago I was in the middle of one of my "Distributed Applications" classes and the professor was explaining us how to """properly""" authenticate users in a server.

To demonstrate how to prevent users from accessing all files under a given directory (i.e. with name "private"), the professor showed us some JSP code that was something like the following pseudocode:

String URI = uri.getRequest();
if (URI.contains("/private/") {
  if (userIsAuthenticated()) grantAccess();
  else redirectToLogInPage();
 }

Of course, I realized what was wrong there, and what could be a security flaw. If an user requests for an URL such as mysite.com/private%2Fmypage then URI wouldn't contain the literal string "/private/" so the user would be granted access.

Yesterday I remembered that class, so as I was curious about it, I indeed found out that not only all my previous web projects in Uni actually had this vulnerability, but even some websites of serious companies also have this issue. You can access to a lot of pages where you should not be able to (basically user profile pages when you haven't logged in), just by percent encoding some character in the URL.

So, as I've realized that there are a lot of people not caring too much about this issue I'm a bit confused. Isn't it a big flaw? What could be some attacks that could be done taking advantage of this issue? I'd just like to know, from a theoretical point of view if possible, how bad could get this issue for a site that does not manage url-encoding properly.

P.S: I know that a plausible solution could be to also check if requested URL is percent-encoded and denying access to it. Or even if we'd like to go further and implement a more secure solution we could use a Key Distributed Center such as Kerberos. However I don't understand why you can find serious websites with this flaw and it doesn't seem soo much of a big deal to them.

Answer 1

What you want to achieve is to enforce some security to /private/ and otherwise serve content freely. Login page shall be outside private.

Servers who handle path based security know how to handle the escapes. What they look for is to bring both the path and the pattern to a "canonical" form. This way, they make sure that they compare slash with slash and not with it's URL escape.

Usually declarative security beats programmed security. This implies that testing path is done in a single place, outside the application, in a 3rd party library written with security in mind. Odds are that the ones who wrote the library were aware about the fact that an URL may contain escapes. Another advantage of declarative security is that it doesn't clutter the code so much. We will not have things like:

delete_entry() {
if (rights.check("delete"))
   actually_delete();
 else {
   log_failed_attempt();
   throw new SecurityException("Not allowed to delete, the attempt has been logged.");
 }
}

A distracted programmer may forget the check for one single action. Rather, have an annotation system of "actions" that will be checked by a caller. If the security handler is missing, fail the call at runtime, this means the programmer has forgotten to "secure" the routine. "The free for all" functions shall have a @freeForAll annotation.

In your case, the server must enforce that /private/ is only served if the visitors have a specific token, which they cannot manufacture themselves. For example, it would be wrong to check for a cookie with the content "authenticated:true", as the clients may generate such cookie without visiting the login.

Answer 2

As stated by @Razvan_P the security component, if based on location, should be something that knows how to handle locations :-).

But that's effectively a very current issue (something like the reflex trick for CTF players). The security 'domain' here is mostly bypassing security filters. All location-based filters are concerned. And you can find issues on that on a lot of HTTP servers (reverse proxy, load balancers), not only applications. Because for most of these agents security is not the main task.

For example :

• admins usually ignore that using rewrite rules with mod_rewrite in apache is not the same for locations and query string arguments (the later is not url-decomposed before running in rewrite rules). And you can find bad (too simple) security rules on online documentations.
• url encoding is applied only on some characters by browsers, but you CAN also url encode any regular character i.e. use %61 for a.
• some characters are always urlencoded by browsers, but not all queries come from browsers, strange things may happen when using low ascii forbidden chars, like the NULL byte or TAB (or backspace, FormFeed, ...)

In terms of security architecture, defense in depth should be applied, and an application should not rely on a previous filtering made only by an HTTP server (like, your PHP application should apply ACL rules, even if you used a mod_rewrite rule in apache httpd).

And to get back the your first question, on the security domain. Some questions around the uri and location syntax errors leads to other domains, not only security filters bypasses, like crlf injection, HTTP Smuggling, or HTTP parameter pollution.

Answer 3

How do you fix this? As Razvan P writes, you need to compare both the URL and the fragment (/private/) in the same canonical form. In practice, this means that you need to URL decode the URL before comparing it, so that %2F is turned into /.

How bad is this? It can be really bad. It basically means that information that is supposed to be private is public. It may also mean that unauthorized persons can modify or delete information. How bad this is depends on what you are protecting, and ranges from slightly embarassing to full on catastrophe.

Why is this vulnerability so common, even on large sites? Well, why do we still see people fall victim to SQLi or XSS, even though it has been known for decades how to deal with it properly? I guess security is complex, humans are falible, and the rest is history.

What do you do now? First of all, you might want to reconsider if poking around on other websites is such a good idea. Depending on the jurisdiction, trying to bypass security filters like that can be illegal. Second, you should consider responsible disclosure for the sites where you did find the vulnerability. Just don't do it in a way that incriminate you.