I can't seem to find an AES tutorial that starts from Unicode to Cipher text. All of the tutorials I found starts with 128 bit block already.

Also, where do I start or what topics do I have to start learning for this IT field (Info. Sec. and algorithms)? Sorry if my post might not be constructive.

up vote 4 down vote accepted

Explanations of AES start with bits because that's what AES, and more general encryption, does: it processes data which is a sequence of bits.

We human beings have been representing information with another mechanism for more than 5000 years, with "glyphs", now often called "characters" (these two terms designate slightly different concepts but let's not obscure the debate). We call that writing. This is in no way a "natural" mechanism, but we are trained to reading and writing from an early age, to the point that we now insist on handling data with that representation.

However, making an encryption system that works on characters has proved difficult, clunky and inefficient. All classic encryption systems until the invention of the computers operated on characters, and all were broken. Computers, on the other hand, use binary because it is much easier and much faster for their silicon brains. Since computers must ultimately interact with humans, various mechanisms have been designed to allow computers to translate characters into bits, and back; collectively, these are called encoding.

Encoding is not encryption, and encryption is not encoding. Encoding far exceeds the scope of encryption. It has its own standards, in particular Unicode. Unicode defines about 120000 distinct characters ("code points" in Unicode terminology) and specifies several methods by which a sequence of code points can be converted into a sequence of bytes (and therefore a sequence of bits); the most common is UTF-8, in which each code point becomes 1, 2, 3 or 4 bytes (basic non-accentuated western letters, sufficient to write most of English words, use only one byte each in UTF-8; Chinese and Japanese ideograms typically take three).

Modern cryptography is designed to work within computers, and any information that fits in a computer must be a sequence of bits, because that's what exists inside a computer. The conversion of such information to and from formats suitable for human consumption, such as encoding of characters, is assumed to have already taken place. This is why documentations on AES do not explain these parts. In particular, there is no encoding of characters that would be specific to the purpose of AES encryption; there is no "UTF-8 for encryption", there is only "UTF-8". Conversely, AES encryption also works on anything else that was encoded to bits, include pictures, music, executable files, or financial data. This is the whole point of working on bits: it works on everything that can be bits, and if it could fit in a computer, then it is bits.

The first step to learning information security is to understand information, and the notion of encoding is the core of it.

  • Doesn't CJK need only two bytes for each symbol? – Mok-Kong Shen Apr 23 '16 at 9:54
  • I am talking about UTF-8 and in UTF-8, code points in the range U+0800 to U+FFFF (which includes Chinese and Japanese) need three bytes each. Other encodings may exist; e.g. UTF-16 would use only 2 bytes for each such code point, but it also needs 2 bytes for each western letter as well. – Thomas Pornin Apr 23 '16 at 13:24

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.