I'm trying to learn a bit more about symmetric encryption and found out that basically every symmetric encryption and decryption is (somehow) in the end based on a simple XOR-Byte-Toggle with using a password and some data.
This is wrong. Let's begin some formal definitions;
A block cipher is a family of permutations where each key is expected to select a unique premutation from the family. And we want the block cipher to be Pseudo-Random permutation (PRP).
Block ciphers are primitives and need a proper mode of operation for the target system. We have tons of mode of operations for block ciphers on which CTR and OFB are common modes to turn any block cipher into a stream cipher. When we define the mode of operation we can talk about
- KPA: Known Plaintext Attack
- CPA: Chosen Plaintext Attack
- CCA: Chosen Ciphertext Attack and its variants as CCA1,CCA2,CCA3,
- and some other more...
We expect a mode that has at least Ind-CPA secure ( Ind -> Indistinguishable) where ECB fails and CTR and CBC can have Ind-CPA security.
The CTR mode actually doesn't need the inverse permutation (i.e. the decryption of the block cipher) so we can use any Pseudo-Random Function (PRF) as a wide range set of functions with better security margins. The CTR mode was originally designed for PRFs.
The CBC mode was very common, see pre TLS 1.3. CBC mode had the padding oracle attacks that are finally removed from TLS 1.3.
Authenticated mode of operation: In TLS 1.3 we have only authenticated modes AES-GCM, AES-CCM, and ChaCha20-Poly1305 where each of them internally uses CTR mode (ChaCha is built-in CTR mode ) and they have also authentication with the GCM and Poly1305. There is no padding in CTR mode to be attacked.
My Question now is, why do we need to make "secure" encryption and decryption algorithms?
Of course, we need. We want to secure our information. For example; If if you insist on using DES with a single key there can be entities that can break your encryption in a day!. The current secure key size is at least 112 according to NIST.
Use AES with a 256-bit key to be secure against even the possible Cryptographic Quantum Computers (QCQ). You can also use ChaCha20 with a 256-bit key ( or better use XChaCha20 with 192-bit random nonces to mitigate a possible (IV,key) pair reuse problem ).
When I get data, which I happen to know is AES (or with whatever algorithm) encrypted and I now want to read it. You need a password to decrypt it. The only way I see a "security" problem here is, that someone who receives that data besides you, needs the password to decrypt it (also knowing that the data is AES encrypted).
The way he now tries to decrypt the data is: Dictionary-Attacks,... (going all the way up to Bruteforce).
Yes, since AES is secure against key searches ( even AES-128 has), the plausible attack is your password. If you use a bad password then the attacker brute-for them starting from the known over 613M pawned passwords. They may try to rainbow tables, too, for all possible combinations like 8 characters.
To mitigate password search attacks you need
- to use a good password with a good strength like generated from dice-wire, minimum 128-bit strength is recommended.
- to use a good Password-Based Key Derivation Function like the latest contest winner Argon2. With the correct parameters, you can reduce the attacker's capabilities. A high number of iterations, memory-hardness, and increased threads are the key to achieving this what Argon2 provides all. Massive GPU, ASIC, and CPU attacks are reduced.
But a computer cannot really define the data to be correctly decrypted as long as the data has no standard header or something how you can identify the correct decryption of it (think of a simple Text-File for example).
This really depends on the encryption scheme that is used. For example
CBC mode has PKCS#7 padding that can be tested during the brute-force as the DES challenger did.
In GCM and ChaCha the authentication tag can be tested to be correct or not.
Note that there are deep details in each like the probability of the padding to be a false-positive and similarly for the GCM and ChaCha. This is not considered here.
If the file is a text file then one can look for the possible strings, too. The more close to natural language the higher candidate to be the true key. All of this can be automated.
With all that in mind, I'm struggling to understand why we not only use a simple XOR-Encryption as it is simple, most likely the fastest way on en- and decrypting files and easy to implement in basically every programming language.
It seems that you are talking about One Time Pad (OTP) encryption. The simple reason is this; to be unconditionally secure ( or call it perfect secrecy or information-theoretically secure) the key size must be equal to the message size. We know this since 1949 by Claude Shannon. This is impractical for today's most systems.
Today we relaxed this condition to be computationally secure against polynomially bounded adversaries. We construct a block cipher or stream cipher that resists known attacks. Then with a stream cipher, we can directly encrypt the messages with x-or. For block ciphers, we need a mode of operation and as mentioned above we select a proper mode of operation for our needs. Different application has different risks and needs a different mode of operations like today we use XTS/XTX mode of operations for disk encryptions instead of CTR mode that was used before and had some attack points.
OTP seems to be easy to implement, however, the key generation and distribution was the common problem. Remember the reuse can cause a crib-Draggin attack and this happened in history,
The more or less key-question here is, what exactly means that an en- and decryption-algorithm is cryptographically secure?
What makes a symmetric encryption algorithm cryptographically secure?
The simple answer is years of research and cryptanalysis during the design.
And remember
There is an old saying inside the US National Security Agency (NSA): Attacks always get better; they never get worse.**
Therefore prepare for all possible attack scenarios even failure of the RSA/ECC to CQC and use post-quantum public-key cryptosystem