when to escape user input

Question

I wonder when is the best way to handle escaping user input.

Two options come to my mind

1) User sends data to server we escape it and then store it into database 2) we store data as it is and escape it when we send data to user.

To me it seems a lot easier escaping and then saving data to database but lets suppose someone finds flow in our website and manages to avoid escaping we have a problem of finding all data that we stored to database un-escaped

on the other hand if we just store data as it is but escape it once we send it to user even if someone finds flow in our website all we have to do is fix bug as our system already assumes that data saved in database in not escaped.

Although second approach seems easier it seems a lot more prone to error. Suppose we generate HTML on server and send it to user and then decide to switch to just sending content to user via ajax, it is easy to forget that we need to escape all the data before sending it to user or implementing new API, or something third.

So I wonder what is preferable way of handling this?

Answer 1

User input is a string. Escaping is done when you want to insert some characters into some HTML / SQL / Whatever code which insists on interpreting some characters into special functionalities. For instance, you have a '<' and you want it to be shown back to the user as a '<', but if you brutally paste the string inside the HTML then the Web browser on the client side will look at the '<' and think that it begins some HTML tag, instead of representing a simple '<'.

In general, you want to keep strings as strings, and delegate any encoding or escaping to specialized functions which do that well. For instance, for SQL, you use prepared statements. With HTML from a PHP context, you would use htmlspecialchars().

The point to notice here is that the kind of conversion, encoding or escaping that you need to perform depends on what you are trying to do with the string. If you need the string to put it in some HTML then you'll use HTML entities (the < for '<' and so on). If you store in the database the already escaped string, then you are betting that you will use the string only by including it in some HTML.

So you should strive to apply encoding/escaping only upon usage. It is more flexible and makes semantics simpler. Within your database, store the string as a string.

Answer 2

EDIT: Luc has pointed out in the concept that I'm unduly slanted towards high-performance solutions. If, in your situation, performance isn't a concern, then it's perfectly acceptable (and preferable, in fact) to store the original data alone and transform it on output. This gives you flexibility to use the data however you need you without maintaining versions.

Original answer below ------------------------------------------------------

To some degree, it depends. First, the answer is rarely to store the raw data and escape it when you read it back out.

The two common solutions are:

1) Escape the data before storing it.

2) Store two copies of the data, one escaped, and one raw.

In virtually any system the ratio of reads to writes is going to be heavily, heavily canted towards reads. It may be 10:1, but it could be 10,000:1. This is why you want to store the data in an escaped format and only parse it when you're writing it, not every single time you want to read it.

The benefit of storing both formats is that the original author can modify the content as intended, you can re-process it if you like, you can review the original data...It gives you some additional flexibility at the expense of a little additional complexity.

This is obviously a bit simplistic, as for instance I'm not considering the effects of caching on the read/write ratio, but it hopefully it conveys the general concept.