Is it risky to URL-encode characters in an already URL-encoded string?

Question

I'm maintaining a legacy web application which is composed of the application itself, accessible to the public, and the REST services which can be accessed only by the application through the LAN. At one point, the application performs a request to a URI of a service which looks like this:

GET https://internal-service/hello/world/user-input

Here, user-input is based on something the user entered, and which is encoded by the application to prevent the user from injecting slashes and other special characters. The encoding is done by calling an appropriate (and reliable) function from the framework.

However, services are separated from the application by a reverse proxy which have a bunch of weird rules. One of them is that there should be no dots in an URI. To cope with this restriction, I replace all the dots from the encoded string by %2E.

For instance, the input: “a/b.c” would lead the following URI to be called:

GET https://internal-service/hello/world/a%2Fb%2Ec

since the first step would produce “a%2Fb.c” and then the second step would replace the dot.

By doing that, am I jeopardizing the security of the whole system, by making it possible for the user to somehow inject unauthorized characters in the URI?

I'm afraid that there might be a quirk related to the way such string would be decoded to an UTF-8 string, i.e. that a specially crafted Unicode character followed by a dot would be encoded to %??%2E which, in turn, would be decoded not to the same character followed by a dot, but by a completely different single character. Or, for instance, I would imagine it to be decoded to a question mark—a character which has a special meaning in an URI.

Is it possible?

Are there other risks?

Answer 1

Adding a URI encoding for a character in an already URI encoded string, when you know where that URI encoded string will be decoded has a vanishingly small probability of exploitability. It would have to allow a string to bypass a filter located between where the string is encoded and where it's decoded.

There are always risks, of course, but you have to specify a receiving end if you want to identify security risks. The biggest risk is that the receiving end won't know to decode the additional URI encoded parts at all, and it will break the system.

If you have no control over the receiving system or insight into what it's going to do with the string, then you can't identify a security risk. There's always the (negligible but non-zero) chance that you'll trip over something.

Answer 2

Are you really asking us to validate your method or just fishing for reassurance that what is happening on the proxy is phenomenally dumb?

It is dumb, by the way. I frequently come across stuff like this invented by some security "expert" which they came up with to address a very specific problem which is usually a failure by someone else to understand how the underlying protocol works.

What you describe might work for your current test cases, but will fail at some point in the future and create costs in the interim.

As an encoding scheme, what you have described here is inadequate and inconsistent.

While you could simply base64 encode the path and query to tunnel this through the proxy, I would like to think there is at least some justification for this madness and such a solution nullifies any protection still in place.

Answer 3

Even if we can't think of a risk, are we sure no hacker will find one?

Why not encode the dots before url encoding? Then we trust the framework's function.