Threat: Mallory enters control characters in a public web application text field that other users might use in a terminal program (via copy+paste or by running a script).

Another threat is a user enters a phishing domain (that is later displayed) and uses a RTL control character so that it looks non-phishy.


is actually http://www.\u202Emoc.lapyap

(Another minor reason is that Postgres barfs if a user enters the null character).

I am not talking about a regular XSS injection that runs javascript. I am using JSoup to prevent XSS and it works fine, but doesn't block control chars (unless they happen to make an XSS)

I looked at OWASP ESAPI Validator but it only validates ASCII chars (ex in the getValidPrintable() method).

It seems like there's no good reason to accept:

\u0000 - \u001F (except carriage return, line feed, horizontal tab)
\u007F - \u009F

Which others should I be concerned with?

Escaping / encoding on save is as incorrect as it is with XSS.

  • Why do you have DBAs copy/pasting stuff into a terminal? That seems to be the bigger problem here, and one which is solved by connecting the database to the application using code (that properly handles all characters). – cpast Apr 7 '15 at 22:39
  • 1
    People make mistakes? New employees run scripts and don't think the content will be malicious? I don't want malicious data in my db if I can avoid it. – Neil McGuigan Apr 7 '15 at 22:44

It is going to be very difficult to blacklist all known bad characters depending on the incoming system, the processing, format changes (HTML, XML, JSON, etc. / ASCII, ebcdic, etc). While there are likely some cases where you need truly "open" input fields, it would be better to define allowed/whitelisted characters on per input field basis.

E.g., if you a requesting a English/romanized name only accept azAZ_ or something similar. If you are dealing with multi-lingual issues, in most cases you should only be expecting one language at a time and could have the user choose an input language and then filter against the required characters in that language in the given context.

When looking at business rules for most apps, there should be few cases where you need to allow full sets and 10 languages at the same time. Allowing full input may not be the best balance of features with security depending on your needs.

Of course, there are still cases like general chat apps and forums so its still a valid question to have certain known blacklists or bad combinations. I am not aware of a master list, so I will leave that to another answer. Control characters are the first thing that come to mind that have little use for most end-user purposes.

The following may be useful reads:

I would imagine some solutions are going to be dependent upon the different systems and applications in your processing pipeline and what they support, and how data is converted, there may not be common answer or approach that is globally applicable.

| improve this answer | |
  • Good advice, thx. The blackhat deck was v useful – Neil McGuigan Apr 8 '15 at 3:48

Accept them all, just encode them into a safe format upon receipt. Furthermore, if you are asking this question, I would first consult an encoding library for a basic implementation before moving on. Encoding can be trickier than you think, and I see custom implementations fail all the time.

In many cases, it is not the character itself that is problematic, but how it fits into a group of characters.

So I shall refrain from giving you list, as it shall be extensive, and not lead you to success. If this is for production, use a peer reviewed code base to do your filtering and encoding. If this is for the purpose of writing a library, study one of those peer reviewed code bases.

Just to be clear: On the way in data should be sanitized, to prevent injection (make sure and use parametrized queries as well.) On the way out data should be encoded, to prevent XSS.

Most modern Web Frameworks have a decent encoder class. If you use Java check out the implementation that OWASP provides: https://www.owasp.org/index.php/OWASP_Java_Encoder_Project

| improve this answer | |
  • I want to prevent possible attack vectors in my DB, not encode stuff on the way out. – Neil McGuigan Apr 7 '15 at 22:31
  • 1
    Secure the data handling between your application and DB. This shouldn't be an issue for your database. – baordog Apr 7 '15 at 22:32
  • @NeilMcGuigan What's wrong with certain characters getting stored in the database? – cpast Apr 7 '15 at 22:38
  • You don't want to encode stuff on the way out. You want to encode it on the way in to protect your database. Also you should parameterize all database queries. – Devon Holcombe Apr 7 '15 at 22:38
  • You know I almost said what Devon Holocombe said. I would say sanitize rather than encode for going in though, wouldn't you think? Edited my answer to reflect this, waiting on the edit queue. – baordog Apr 7 '15 at 22:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.