Take the 2-minute tour ×
Information Security Stack Exchange is a question and answer site for Information security professionals. It's 100% free, no registration required.

Why shouldn't we create our own security schemes?

I see a lot of questions around here about custom crypto and custom security mechanisms, especially around password hashing.

With that in mind, I'm looking for a canonical answer, with the following properties:

  • Easy for a newbie to understand.
  • Clear and explicit in why rolling your own is a bad idea.
  • Provides strong examples.

Obligatory xkcd.

share|improve this question
1  
For reference, D.W. posted a great starting point here. –  Polynomial Aug 6 '12 at 15:41
2  
A couple people that thought they knew enough got together and created an encryption scheme called WEP for wireless networks. You can crack WEP encryption in just a few minutes. A "roll your own" methodology was used. Read about it, I'm just reminding you of this (you probably already know). –  Everett Aug 7 '12 at 1:55
2  
@Andrew Smith But will the time and energy yield crypto that is better than what is in the public space for free? It sounds like you're trying to hide behind a security by obscurity approach. I don't say that in a mean way, I'm just trying to understand why you would spend resources to reinvent the wheel, and not have nearly as much testing done on the design you pay for. –  Everett Aug 7 '12 at 18:09
5  
@AndrewSmith Sorry, but when every security expert I've ever spoken to or read articles by disagrees with you, I cannot take your views as having any merit. Security by obscurity has been proven to be absolutely devastatingly broken as a concept, both in terms of theory and practice. Modern cryptography that has undergone stringent peer review over a long period, with proper security proofs and complete mathematical analysis, is the only guarantee of any real cryptographic security. –  Polynomial Aug 7 '12 at 21:18
4  
@Polynomial: I think you may have been conversing with a Markov chain generator. –  Piskvor Aug 9 '12 at 9:08

5 Answers 5

up vote 47 down vote accepted

You can roll your own, but you probably will make a major security mistake if you are not an expert in security/cryptography or have had your scheme analyzed by multiple experts. I'm more willing to bet on an open-source publicly known encryption scheme that's out there for all to see and analyze. More eyes means less likely that the current version doesn't have major vulnerabilities than something developed in-house by non-experts.

From Phil Zimmermann's (PGP creator) Introduction to Cryptography (Page 54):

When I was in college in the early 70s, I devised what I believed was a brilliant encryption scheme. A simple pseudorandom number stream was added to the plaintext stream to create ciphertext. This would seemingly thwart any frequency analysis of the ciphertext, and would be uncrackable even to the most resourceful government intelligence agencies. I felt so smug about my achievement.

Years later, I discovered this same scheme in several introductory cryptography texts and tutorial papers. How nice. Other cryptographers had thought of the same scheme. Unfortunately, the scheme was presented as a simple homework assignment on how to use elementary cryptanalytic techniques to trivially crack it. So much for my brilliant scheme.

From this humbling experience I learned how easy it is to fall into a false sense of security when devising an encryption algorithm. Most people don’t realize how fiendishly difficult it is to devise an encryption algorithm that can withstand a prolonged and determined attack by a resourceful opponent.

(This question has more discussion of the above quote.)

If you are not convinced of "Don't Roll Your Own [Cryptography/Security]", then you probably are not an expert and there are many mistakes you likely will make.

Is your application robust against:

  • Timing Attacks. E.g., to the nanoseconds do completely-bad keys and partially-bad keys take the same amount of time in the aggregate to fail? Otherwise, this timing information can be exploited to find the correct key/password.

  • Trivial Brute Force Attacks; e.g., that can be done in within seconds to years (when you worry about it being broken within a few years). Maybe your idea of security may be a 1 in a billion (1 000 000 000) chance of breaking in (what if someone with a bot net tries a few billion times?). My idea is to aim for something like 1 in ~2^128 ( 34 000 000 000 000 000 000 000 000 000 000 000), which is roughly ten million billion billion times more secure and completely outside the realm of guessing your way in.

  • Attacks on user accounts in parallel; e.g., you may hash passwords with the same (or worse no 'salt') on all password hashes in the db like what happened with the leaked linkedin hashes.

  • Attack any specific account trivially simply. Maybe there was a unique random salt with each simply hashed (e.g., MD5/SHA1/SHA2) password, but as you can try billions of possible passwords on any hash each second, so using common password lists, dictionary attacks, etc. it may only take an attacker seconds to crack most accounts. Use strong cryptographic hashes like bcrypt/PBKDF2 to avoid or key-strengthen regular hashes by a suitable factor (typically 10^(3-8).

  • Attacks on guessable/weak "random" numbers. Maybe you use microtime/MT-rand or too little information to seed the pseudo-random number like Debian OpenSSL did a few years back.

  • Attacks that bypass protections. Maybe you did hashing/input validation client side in web application and this was bypassed by the user altering the scripts. Or you have local application that the client tries running in a virtual machine or disassembles to reverse engineer it/alter the memory/ or otherwise cheat somehow.

  • Other attacks, including (but not attempting to be a complete list) CSRF, XSS, SQL injection, network eavesdropping, replay attacks, Man in the Middle attacks, buffer overflows, etc. Best protections very quickly summarized. CSRF: require randomly generated CSRF tokens on POST actions; XSS: always validate/escape untrusted user-input before inputting into db and displaying to user/browser; SQLi: always use bound parameters and limit how many results get returned; eavesdropping: encrypt sensitive network traffic; replay: put unique one-time nonces in each transaction; MitM: Web of Trust/Same as site last visited/Certificate issued by trusted CA; buffer overflows: safe programming language/libraries/executable space protection/etc).

You are only as strong as your weakest exploitable link. Also just because you aren't rolling your own scheme, doesn't mean your scheme will be secure. Its quite likely that the person who created what you rolled out was not an expert, or created an otherwise weak scheme.

share|improve this answer
1  
The biggest problem I notice among beginning coders, is that they don't take security risks seriously. If I mention what kind of weaknesses some system has, the common response is rolling their eyes... The attacks are real and do happen, and if you can't even apply proper security at school then how's your application supposed to fare when it's live for hundreds of customers? :/ The only way to convince them is usually providing examples, and best of all a live demo. So if you have examples... I think it'd be a nice addition! –  Luc Sep 21 '12 at 19:41

The example cipher Phil Zimmermann cited: A simple pseudorandom number stream was added to the plaintext stream to create ciphertext, is easily thwarted. Some asked why. Here is why:

  1. Step 1, feed the cipher plaintext of all 0's. Since 0 added to any number is same as the number un-changed, it completely exposes the pseudorandom number stream in the output.

  2. Step 2, given what you discover in step 1, and given the encrypted text of a secret message, you can easily calculate the original text by simple subtraction of the pseudo random number stream which you now know.

With that said, I disagree with the general notion that inventing your own cipher is almost bad. Any cipher algorithm, upon first creation, has a status of UNKNOWN security strength, even those created by experts. The experts have the advantage of being able to immediately apply typical known attacks and see how the new cipher stands, and be able to quickly rule out some inventions as weak, and amateurs do not have that ability. But it does not necessarily mean the cipher is weak. It means the strength is unknown and it may well be a very strong cipher. You just do not known.

For example I am not an expert. Here is a trivial cipher that I came up in 5 seconds without much thought:

  1. Pick a random number of 128 bits as your key.
  2. circular Rotate the key by X number of bits, Call the result value A.
  3. Circular rotate the key by Y number of bits, split it into two 64 bit parts and multiply them to produce a value B.
  4. XOR value A and B to produce a new key, use only bit 0 to bit N-1 of this key to encrypt N bit of plain text by XORing.
  5. Go back and repeat steps 2,3,4.

This seems to be simple enough, and I have not done any analysis to see how strong it is. But all I can say is this has an unknown security strength. It may well be very strong. If you immediately jump into conclusion to say it is weak or unreliable, I challenge you to come up with a method to crack it. Can you?

I think when it comes to secrucity strength, there are only two: 1. Unknown security strength 2. Known weak security strength

There is no such thing as known strong security strength. If the scheme has been scrutinized a lot without finding a crack, it only means no crack has been found yet and the status remains to be unknown, and it does not necessarily mean strong, because a crack may well be discovered tomorrow.

share|improve this answer
5  
This is a great example of why you shouldn't roll your own cypher: it has exactly the same weakness to chosen-plaintext attacks that Zimmermann's cypher does. Feed in a plaintext of all 0s that is at least as long as the cyphertext you are trying to decode, and you get the keystream used to encode the cyphertext. XOR that with the unknown cyphertext, and you get the original message. –  Mark Jul 24 at 1:09
1  
Anthony - your final paragraph is correct, however your example is provably weak despite you thinking it was effective. This just helps to support the principle, which is: don't do it, unless you are an expert. And even if you were an expert, don't do it alone. –  Rory Alsop Jul 24 at 7:20

Bruce Scheier wrote back in 1998:

Anyone, from the most clueless amateur to the best cryptographer, can create an algorithm that he himself can't break. It's not even hard. What is hard is creating an algorithm that no one else can break, even after years of analysis. And the only way to prove that is to subject the algorithm to years of analysis by the best cryptographers around.

Cory Doctrow dubbed this concept "Schneier's Law" in a 2004 speech as follows:

Any person can invent a security system so clever that she or he can't think of how to break it.

As a follow-up, this again from Schneier:

When someone hands you a security system and says, "I believe this is secure," the first thing you have to ask is, "Who the hell are you?" Show me what you've broken to demonstrate that your assertion of the system's security means something.

Phil Zimmmerman also wrote in his original PGP papers:

When I was in college in the early seventies, I devised what I believed was a brilliant encryption scheme. A simple pseudorandom number stream was added to the plaintext stream to create ciphertext. This would seemingly thwart any frequency analysis of the ciphertext, and would be uncrackable even to the most resourceful Government intelligence agencies. I felt so smug about my achievement. So cock-sure.

Years later, I discovered this same scheme in several introductory cryptography texts and tutorial papers. How nice. Other cryptographers had thought of the same scheme. Unfortunately, the scheme was presented as a simple homework assignment on how to use elementary cryptanalytic techniques to trivially crack it. So much for my brilliant scheme.

share|improve this answer
    
It would be really nice to know what that "introductory cryptography text" is :) –  user54609 May 8 '13 at 15:19

The original post asked for an example:

There is a good story of a bad cryptosystem causing problems. Mary Queen of Scots was imprisoned by her cousin Queen Elizabeth I and was communicating with people on the outside via encrypted letters. The alphabet was replaced with a cryptoalphabet of squiggles, crossed circles and triangles with extra letters assigned for common letters, like e,t,i and o so the meaning of letters could not be found quickly by frequency analysis. They also added a few null characters which were ignored on the decryption to throw off analysts. The problem was that The Queen had a very competent cryptanalyst on her staff in the person of Thomas Phelippes who was able to decrypt the messages as they were intercepted.

As things progressed, Mary went along with a plot to have her escape and take over the throne. When The Queen’s agents intercepted the last letter from Mary before passing it on, they added an encrypted sentence asking for the names of those involved in the plot “so they can be properly rewarded.” Mary’s correspondent dutifully answered and The Queen’s agents had everybody involved executed.

When my kids were small, I would send them cryptograms with their lunch (with a key (using a Vernam cipher)). Generally, they were jokes but they were never of any importance. In a case like that, roll your own is fine. If you are plotting to overthrow The Queen of England (or the Shah of Iran or the slowly reforming Thug-ocracy in Myanmar), I would suggest that you make sure that what you using cannot be easily decrypted. I think it was Bruce Schneier that said, anybody can come up with a cryptosystem they cannot decrypt but coming up with one nobody else can decrypt is harder.

share|improve this answer

There is a house in my area with a really nice deck outside the second story family room. It looks swell, until you go underneath and see how it was constructed. It seems the homeowner decided he did not need to pay big bucks to a builder or architect to tell him how to build a deck. He built it himself and it looks like a chaotic spider web of 2x4’s underneath. It PROBABLY will be fine. Personally, I would rather not risk life and limb on an amateur construction job like that.

I think that if you want to develop an algorithm to do encryption, you should do so and have a good time of it. I would not recommending using it to hide your online bank statements but if you want to encrypt your girl friend’s love letters on your home computer, that should be fine—provided your wife is not a cryptanalyst.

There is a story in “The American Black Chamber”* about the Navy developing their own ciphers. The Navy would show their new cryptosystem, pleased with themselves and Yardley, the Army analyst, would promptly break the code, explaining what they had done wrong. They would offer to fix the code but Yardley pointed out that while they could fix specific weaknesses, without a solid understanding, they were going to always have a problem. Their system was intrinsically flawed. It is a little like patching a leaky roof. You can patch forever but the water is still going to find its way in. If you don’t want to get wet, the roof needs to be constructed by somebody that knows more than a little about roofs.

Did I ever tell you about the do-it-yourself brain surgery I performed on my late mother-in-law? Everything went fine until she went and died. Seriously, few of us would trust our health to an amateur doctor; do you really want to trust your secrets to amateur software? I hate to admit it but I buy a lottery ticket every few months. I fully expect to lose but the potential payout is huge. I can play the odds and maybe I will come out ahead. If I don’t, I am out a buck. Why play the odds on encryption? The payout is not there.

Regards, /Bob Bryan

  • Recommended: Herbert O. Yardley, “The American Black Chamber” -- A book as interesting today as when it was written in 1931. “The American Black Chamber was filled with good stories well told, as well as frank descriptions of Yardley's successes in cryptanalysis. It was a best-seller in 1932 -- overseas as well as domestically.” From NSA: Pearl Harbor Review - The Black Chamber
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.