The protocol is actually like [data, HMAC-SHA1(data)] (or with another hash instead of SHA-1). What is needed here is to authenticate the data, i.e. to guarantee that it comes from a source that knows a particular secret key. A hash allows the recipient to check that the data is equal to something else, but the recipient has nothing to compare to. In contrast, a MAC allows the recipient to verify that it was generated by an entity that knows the secret key.
You're trying to build a MAC on top of a block cipher. This is possible, but you're doing it wrong. Let's assume, as you do, that the data is encrypted with a block cipher in CBC mode.
The data and checksum itself are encrypted, and any small change in the encrypted data would change at least one entire block (unpredictably by a possible attacker).
That's true: one block changes unpredictably. However, this is not good enough: you need to look at the consequences on the other blocks.
Due to encryption mode (CBC) and the fact, the hash is stored at the end of the packet, I suppose even the hash would change unpredictably.
No, the hash would not change unpredictably if the attacker is careful. It is in fact pretty easy to avoid changing the hash you chose, which is the XOR of all the blocks. CBC uses XOR internally, so this shouldn't come up as a complete surprise.
Consider a message consisting of three blocks P₁, P₂, P₃ and the checksum P₊ encrypted with CBC, and let C₁, C₂, C₃, C₊ be the corresponding ciphertext. I'll write E for the block encryption function and D for decryption.
C₁ = E(P₁ ⊕ IV)
C₂ = E(P₂ ⊕ C₁)
C₃ = E(P₃ ⊕ C₂)
C₊ = E(P₊ ⊕ C₃) where P₊ = P₁ ⊕ P₂ ⊕ P₃
The receiving side decrypts with
P₁ = D(C₁) ⊕ IV
P₂ = D(C₂) ⊕ C₁
P₃ = D(C₃) ⊕ C₂
P₊ = D(C₊) ⊕ C₃
A man-in-the-middle attacker flips C₁ and C₂. Here's what the recipient sees:
P₁' = D(C₂) ⊕ IV
P₂' = D(C₁) ⊕ C₂
P₃' = D(C₃) ⊕ C₁
P₊' = D(C₊) ⊕ C₃
Notice that P₊' = P₊ — perturbing a CBC ciphertext only has an effect up to the next block after the perturbation, the rest remains intact. And
P₁' ⊕ P₂' ⊕ P₃' = D(C₂) ⊕ IV ⊕ D(C₁) ⊕ C₂ ⊕ D(C₃) ⊕ C₁
P₁ ⊕ P₂ ⊕ P₃ = D(C₁) ⊕ IV ⊕ D(C₂) ⊕ C₁ ⊕ D(C₃) ⊕ C₂
Oh, look, these are the same. The checksum on the modified message is correct.
It is possible to design a MAC algorithm based on CBC, but you have to be more careful than this. A naive CBC-MAC allows attacks of the sort you noticed, by playing with the length. There are variants that don't have this problem. However, there's a snag: you must not use the same key for CBC encryption and CBC-MAC! This is because if you use the same key, and the attacker has control over some of the plaintext, then the attacker can cause the sender to calculate a MAC for her. For example, the CBC-MAC of a one-block message is just the result of encrypting that block. So if the attacker wants to send a one-block message, all they have to do is to arrange for the sender to encrypt a message that starts with that block and read the result. It is a general principle that using the same key for two different things is dangerous because it allows protocol confusion errors where the same calculation produces data that must be public for one of the uses of the key and private for the other use.
Xor is not as good as a MAC. In fact, even your initial reading of the authentication tag as [data, SHA1(data)] wouldn't work, because CBC encryption of a hash is not secure (the flaw is more subtle than with xor, but it exists).
