I just read this article, which is a top 10 list of developer crypto mistakes. In number 7, he states that "HMAC should not be applied to the plaintext but instead to the ciphertext combined with the IV." I always thought that the HMAC was computed from the plaintext. After doing some more reading, I realised that my initial thoughts were incorrect. However, I still do not understand why the IV needs to be concatenated to the ciphertext before applying the HMAC. Why is this the case?
This sentence applies specifically when HMAC is used in combination with encryption. HMAC can be used alone if you only care about message authenticity: it guarantees who produced the message, but does not hide the content of the message. This sentence does not apply in this case. When HMAC is used for authenticity alone, the HMAC is calculated over the plaintext and that's that.
When you want to have both secrecy and authenticity, i.e. when you want to both hide the content of the message and guarantee its origin, then you need authenticated encryption. There are modes of operation that provide authenticated encryption; GCM, which is mentioned in the article, is a popular choice and a good choice.
It's also possible to cobble an authenticated encryption mechanism by combining an encryption mechanism (such as a block cipher in CTR or CBC mode) with an authentication mechanism (such as HMAC). However, doing this correctly is subtle. There are two main ways to do it: encrypt-then-MAC or MAC-then-encrypt. The sentence you quote says that you should do encrypt-then-MAC, i.e. first encrypt the message and then authenticate the end result (which is the ciphertext including the IV). This is a good recommendation in principle, but it's not easy to implement it correctly.
The recommendation against GCM is a bad one. Yes, it fails if the IV is reused. However a decent crypto library will call the RNG on its own to generate an IV, eliminating the risk of IV reuse (unless the system RNG is broken, in which case your key generation is also broken). (There are bad crypto libraries that require the caller to do all the plumbing; avoid them.) It is a lot easier to use a GCM library correctly than to assemble HMAC and encryption correctly. The unclearly-worded recommendation to do encrypt-then-HMAC contradicts the article's very own recommendation #1 against “crypto soup”.
Unless you have a very good reason to do so, and you understand perfectly what you're doing, always use the highest-level crypto libraries that do what you need. Use a protocol layer that relies on TLS and performs the certificate verification if possible, use the operating system's file or disk encryption mechanism and password storage instead of rolling your own, use cryptographic libraries which do all the RNG calls and IV generation and appending internally if you have to go down to the crypto level, etc.
Changing the IV will result in a predictable tamper in the first decrypted block.
P' = P xor IV xor TamperIV.
Since you are trying to prove recovery of the original data you need to include the IV.
And you need to apply the HMAC to the ciphertext+IV instead of the plaintext for two reasons:
- The same data with the same key will HMAC the same, which may disclose data (defeating the random IV in CBC mode)
- If someone is tampering with the encrypted data they may be trying to get you to side channel reveal data, or maybe crash. Proving that the ciphertext (and IV) are unaltered says it's safe to decrypt.