Hiding sensitive data in URIs

Question

This is a potentially hypothetical question since it has not been determined whether we will be required to do this but I figure it's a question that will come up more often.

Background

When implementing RESTful services the standard approach is that the URI contains the information required to identify the resource in question. While I am not a restafarian there are a lot of advantages to this approach over that we wish to take advantage of.

Problem

We are subject to regulatory requirements around protecting certain types of information. Some of these protected elements will end up in resource paths if we follow the basic REST formula.

TLS will cover most of the concerns about because the URLs are encrypted. However, there are a few places where they may be stored in the clear. Notably they are often written to access logs (which is useful) by default and if people use browsers for this, they can be stored in the clear in their browsing history.

Approach

Assuming this is an issue and we want to use such an approach it seems there is one prime contender for the solution, which is to encrypt parts of the URI or all of it. Hashing does not seem to be an option since the data is highly predictable and calculating the entire set of values is easy. Encrypting the entire URI will limit a lot of the advantages of the approach. Symmetric encryption requires distributing a secret key widely and therefore defeats the purpose. The option that seems the best is to encrypt on the sensitive data with a public key.

As I understand it, your encrypted output needs to be at least as long as the modulus. So if we go with a 2048-bit key, each the data will need to be at least 256 bytes. This will result in some fairly long URIs but I think that should be OK for a while anyway and that limit may not really apply to our solution.

Does this approach seem valid and/or is there anything else I am missing? Would it be valid to use the same key-pair that would be used for TLS? There probably needs to be some other information in the encrypted portion to avoid replay attacks or mapping an encrypted value back to given key, correct?

NOTE: I should mention that for my specific problem, only authenticated parties would be able to call. This is not something that we mean to be expose on the public internet although mistakes can happen. All parties involved are legally-bound to protect the data. I think it's worth exploring more general usage of such an idea because of the growing popularity of RESTful services.

Answer 1

it seems there is one prime contender for the solution, which is to encrypt parts of the URI or all of it. Hashing does not seem to be an option since the data is highly predictable and calculating the entire set of values is easy. Encrypting the entire URI will limit a lot of the advantages of the approach. Symmetric encryption requires distributing a secret key widely and therefore defeats the purpose. The option that seems the best is to encrypt on the sensitive data with a public key.

Your logic is sound.

I know of one product that does exactly as you suggest; sensitive data is encrypted in the end-users browser by Javascript code, and an HTTPS GET request is made in which some of the 'key=value' parameter pairs have a public key encrypted, base64 encoded string.

The advantage is that these requests can be passed across networks with multiple TLS endpoints without the sensitive data being subject to decryption, and any web server logs or proxies in the middle won't have access to the sensitive data.

Does this approach seem valid and/or is there anything else I am missing?

It is valid, but here are the things you don't want to miss:

1. You need to have a client with enough intelligence to perform the encryption for you. That means Javascript, or a custom app, or something.
2. Don't roll your own crypto. Use a solid, reputable library and get a code review of your app.
3. Key management. You need to generate keys, publish keys, rotate keys, and retire keys... and you need to do it carefully, because to do it right means to balance between losing control of the data and losing access to the data.

Would it be valid to use the same key-pair that would be used for TLS?

I would recommend against it. Aside from the general rule-of-thumb of "don't overload cryptosystems," there are some situations where that key isn't "trusted". One that I've run into is that DDoS mitigation services often like to have a copy of your web certificates and keys if they're fronting your traffic during an attack.

There probably needs to be some other information in the encrypted portion to avoid replay attacks

That depends; if the REST transaction you're working with is idempotent, then it's not that important. If it does change something, then yes, replay protection is more important.

or mapping an encrypted value back to given key, correct?

Yes, it's not absolutely necessary - depending on how many keys you use and how long their OUP and RUP tail is - but it's much easier not to have to guess which key :)

Answer 2

I was reminded of this because it came up as a "popular question." Since I asked this I have learned that the OWASP recommendation is to use request headers:

• In POST/PUT requests sensitive data should be transferred in the request body or request headers
• In GET requests sensitive data should be transferred in an HTTP Header

While this makes the API somewhat more difficult to use, there is support for this approach in at least some common frameworks.

Answer 3

Does this approach seem valid and/or is there anything else I am missing?

Your approach is valid, however; I wouldn't dismiss using a preshared symmetric key for the following reasons.

• Asymmetric is far slower to decrypt than symmetric. If the service has to decrypt multiple values from your URI performance could be impacted.
• The attack vector for this encryption is limited. TLS should be protecting this data in most cases.
• Different consumers can be given different pre-shared keys for added security (addressing your concern with widely shared symmetric). This also allows for more flexibility with key rotation.
• Asymmetric encryption will result in a large encrypted text. If your URI + querystring are long enough it could possible cause a 414 (Request-URI Too Long)

Would it be valid to use the same key-pair that would be used for TLS?

In theory you shouldn't because if an attacker obtained your key-pair they could decrypt both your TLS connections & your encrypted headers.

... Now from a realistic/support standpoint, if your TLS key-pair is obtained that's a doomsday scenario. Having a second keypair would not prevent a "full breach" event since you'd likely have sensitive data in the headers/body that could be decrypted.

I think you need to carefully weigh the options for your specific business need.

Answer 4

The more standard approach is to move sensitive information into the body.

Really, this comes down to your logging practices. You could make the above suggestion worthless by including the body in your logs, or you could sidestep the issue by not logging the full URI. The information is always still available to your server (even in your encrypted scheme, it has to be decrypted by at least some part of the stack).

It seems to me that, rather than implementing a complex scheme that bloats URLs, makes it harder to debug, and causes users to wonder what's happening, you'd be better off just changing the defaults on your logs to something that suits your environment.

Answer 5

There are a few points that your approach hasn't considered:

• encrypt parts of the URI or all of it

  How will you encrypt the URI? The browser does it or the web-page? If the browser does it, how does the browser know what part of the URI to encrypt and if the web-page does it, how will you get the TLS certificate of the server? Read this. Also, you can't encrypt the entire URL otherwise the browser will not know the server that it should talk to.

• There probably needs to be some other information in the encrypted portion to avoid replay attacks or mapping an encrypted value back to given key, correct?

  If you are using TLS, then the replay detection is already handled by TLS (Are SSL encrypted requests vulnerable to Replay Attacks?).

There are several discussions about encrypting URL parameters that have happened on StackOverflow: 1, 2 and some other google search results: It's a bad idea, how to do it in Java