Take the 2-minute tour ×
Information Security Stack Exchange is a question and answer site for Information security professionals. It's 100% free, no registration required.

. . .better solutions have been proposed and will be worth considering once they have withstood the test of time (i.e. “5 to 10 years in the field, and not broken yet”).

-From Thomas Pornin's excellent blog post on password hashing.

The quote seems to me to be a case of the chicken and the egg. New solutions need field experience, but until they get field experience they cannot be trusted, and hence cannot be used in the field.

The AES selection process somewhat avoided the problem by bringing a great deal of academic force to bear. Over five years, the best cryptographers in the world meticulously examined the submitted algorithms, with the result that Rijndael was chosen as the best compromise between security, speed, and usability.

But AES had the significant advantage that the standard it was replacing was demonstrably weak against modern computer hardware and in dire need of replacement. Most new protocols or algorithms don't have this hard limit to help them achieve prominence.

Given that the standard answer to "which algorithm?" or "which method?" is always "use the trusted", how do new solutions ever get their field experience? Does it rely on someone taking a leap of faith with their sensitive data, or does it rely on use of the new solution with non-sensitive data and then extrapolating based on that experience, or what?

(I understand that the quote's context was specifically related to password hashing. If generalizing from that to all security methods is unfair or off-base, knowing why would be useful.)

share|improve this question
    
The same problem exists for all technologies - a better solution is usually available, but no one uses it because no one else uses it. And the solution tends to be the same too - either everyone agrees to a standard, or some big player in the field starts supporting it and convinces their customers it's what they want, at which points all their competitors will follow suit. –  BlueRaja - Danny Pflughoeft Sep 4 '13 at 19:38

2 Answers 2

up vote 4 down vote accepted

The bootstrap process is achieved when people do use the new algorithm. To gain trust in the algorithm's strength, it must have exposure. Exposure is achieved through either formal competitions like for AES, or, more recently, for SHA-3; or it is achieved through the algorithm being widely deployed and becoming an interesting target.

Open competitions are tricky: in order to claim a good level of exposure, the competition must have attracted a lot of researchers for quite some time, and that's not easy because academic researchers rarely work for money, but for glory (or at least what passes as fame in academic circles). It needs a powerful sponsor. NIST is good for that, and did it well for AES; arguably, SHA-3 was also a success, although the end of the process has a somewhat bitter aftertaste (because NIST is currently deciding what tweaks it will push into the winner algorithm, and such tweaks undermine the whole trust building process). There were other "competitions" such as eSTREAM for stream ciphers; it did get a fair amount of exposure, but "not as much" as AES or SHA-3. For password hashing, there is an ongoing competition which is at the candidate collection phase (until January 31st, 2014).

Wide deployment occurs when some organization decides to deploy the algorithm "early", because the new algorithm offers a decisive usability advantage (e.g. a signature algorithm which has very short signatures, or requires very little computing power), or because the powers-that-be in that organization feel (wrongly) that "newer is better" and want to be early adopters of the new nifty cryptographic technology. For instance, SSL was widely deployed because it filled a niche which was not otherwise covered, even though the protocol was really not up to known "good practices" at that time (SSL 2.0 was quite bad and lead to a complete rewrite as SSL 3.0 less than two years afterwards).

Thus, as cryptographers, we MUST recommend that people refrain from using algorithms which are "too new" and "insufficiently reviewed", but we are also quite happy that some people do the "wrong thing" and do not refrain; it gives the exposure which is necessary for the bootstrap, and also ensures that cryptographers won't get out of job (a research article which demolishes a poorly-specified protocol -- WEP, I am looking at you -- is still a published research article). The best we can do, in the meantime, is to prepare: cryptographers can design algorithms based on rational constructions which thus stand a reasonable chance of surviving the deployment. For instance, when bcrypt was first described, it had not (of course) been exposed in the wild for several years, but at least the authors tried to follow some justified rationale, and the algorithm turned out to be "mostly good".

share|improve this answer

Disclaimer: I am not a cryptographer. Someone *ahem* Bear *ahem* will probably write a more complete answer. I will like to just point out a few points.

But AES had the significant advantage that the standard it was replacing was demonstrably weak against modern computer hardware and in dire need of replacement. Most new protocols or algorithms don't have this hard limit to help them achieve prominence.

You don't need a hard limit to encourage new algorithms to be studied. For example, NIST recently concluded a competition to find a new SHA-3 algorithm. This has proven to be quite successful in encourage cryptographers to submit and study new algorithms. However, SHA-2 is still secure by all existing measures.

Given that the standard answer to "which algorithm?" or "which method?" is always "use the trusted", how do new solutions ever get their field experience? Does it rely on someone taking a leap of faith with their sensitive data, or does it rely on use of the new solution with non-sensitive data and then extrapolating based on that experience, or what?

New algorithms do not necessarily have to be deployed in mission critical applications to be "studied". The process of coming up with new algorithms usually involves publishing a paper on the algorithm and having endless back and forth arguments with other cryptographers about it's merits. (Note: I'm not a cryptographer, I'm just assuming :P). Once enough bickering has occurred, an algorithm can be deemed either "secure" or "broken" in a mathematical sense.

If an algorithm has been deemed "secure", the next step for the algorithm will be getting good, easy to use implementations written for popular programming languages. This is crucial. Implementation matters and it isn't easy to get right. Only after this has been accomplished can an algorithm be deemed worthy of use in the field. There will probably be a whole process of bugs being found and fixes being introduced. The history of SSL/TLS is a good example of this.

The first part is easy. New algorithms are introduced and studied frequently in an academic sense. The second part less so, you are right in saying that getting new algorithms and protocols tested in the field is a chicken-and-egg problem. This unfortunately, is a fact of life. Why fix what isn't broken (yet)?

share|improve this answer
    
Right. I understand the academic attacks against new published solutions. The problem is, academic study almost never covers exactly the same ground as malicious attackers will. (Incentives being what they are.) Your last paragraph is pretty much what I'm getting at with this question. –  Jonathan Garber Sep 4 '13 at 16:31

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.