48

I just learned a few things about hashing algorithms – MD5 and SHA-1. So, if I am not wrong passwords are hashed so that in a rare situation of your database being compromised a hacker can not see all the passwords in your database as the passwords are hashed. MD5 is not safe anymore as most of the common passwords and their hashed values are out on the internet. And hence people now use other hashing algorithms.

So, my questions are :

  1. Is hacking a database that easy? I mean instead of finding new ways to hash, why can't they make their database more secure or "hack-proof"?

  2. If a hacker manages to hack into some database, why would he want to know the hashed passwords? I mean he can change other columns of data. For example, he could search for his username and increase his bank balance. So, shouldn't all the fields be hashed?

migrated from crypto.stackexchange.com Oct 28 '16 at 22:34

This question came from our site for software developers, mathematicians and others interested in cryptography.

  • 38
    You have a misunderstanding about what MD5 is unsafe. See here. It's about speed, not published results. (The latter can be overcome with a long, random salt.) SHA-1 suffers from the same problem. – jpmc26 Oct 28 '16 at 23:48
  • 42
    A lot of times hackers don't actually manage to "hack into the database", that is, hackers don't manage to get access to the database. Instead they just manage to dump the database. So they can't change it but can get the hashed passwords. So they "why would he want the hash" is because that's what he gets. From there he would want to try to reverse the hash using rainbow tables etc to get at least a few passwords that match some of the hashes. Then he can login. Note that he would almost NEVER be able to get all the passwords. Only some. But then he would be able to login. – slebetman Oct 29 '16 at 1:32
  • 34
    I really wish my bank would hash my balance. I'd find a stupidly-high numeric collision for the hash and buy the world. – ceejayoz Oct 29 '16 at 23:42
  • 4
    @tsukumogami Yeah, but OP mentions MD5 and SHA1. So, I'm a budding multi-quintillionaire! – ceejayoz Oct 30 '16 at 1:20
  • 2

11 Answers 11

59

Some things first:

  • Forget about MD5 immediately. It's old and weak.
  • Ideally, forget about SHA1 too. There are SHA2 and SHA3.
  • This hash algorithms in their pure form are useful for many things, but not for passwords. Use them in combination with eg. PBKDF2, or use bcrypt (and don't forget salting).

So, if I am not wrong passwords are hashed so that in a rare situation of your database being compromised a hacker can not see all the passwords in your database as the passwords are hashed.

Correct. While this won't help making your server more secure (after all, it's already compromised in such a situation), it prevents eg. the attacker using the passwords on other sites. Sadly most people use one password for multiple things.

Is hacking a database that easy? I mean instead of finding new ways to hash, why can't they make their database more secure or "hack-proof"?

You could (and you should). But:

  • Even with all your efforts, there always is some chance the attacker can get access. Bugs in software and devices you don't know about and/or you can't fix, and many more things.
  • Often you can't give your best because of managers who don't think security is important, have no budget for it etc.

If a hacker manages to hack into some database, why would he want to know the hashed passwords? I mean he can change other columns of data. For example, he could search for his username and increase his bank balance. So, shouldn't all the fields be hashed?

If all fields are hashed, the database is useless for you too; so you can't do that. Remember, a good hash can't be reversed.

As said above, as soon as your DB/Server is compromised, the damage for you already happened. The hashed passwords just prevent that the attacker gets access to other accounts of your users too (and then some users would sue you, so hashing helps you too).

  • 65
    @axapaxa Wait, what? argon2 is only 1 year old and there's already two published attacks on it, and scrypt is demonstrably weaker than bcrypt for the most common applications. There is absolutely nothing wrong with recommending bcrypt, even in 2016. – corsiKa Oct 29 '16 at 2:08
  • 33
    @axapaxa Cryptography (ie. math) doesn't become obsolete just because it's old. In fact it gets stronger as more people examine it and fail to find flaws. Blowfish, as used by bcrypt, remains secure. bcrypt is further designed to avoid obsolescence by allowing you to scale the cost of the hashing function. bcrypt users can defend against increasingly powerful brute force attacks or future potential flaws in Blowfish by increasing the scaling factor. – Schwern Oct 29 '16 at 5:56
  • 28
    To add to the "why" of hashing: Even in a perfect security scenario, you still want to hash, because nobody except the user themself has to know that password. DBAs and other legitimate eyes on the database should never see the user password. – Darkhogg Oct 29 '16 at 8:17
  • 29
    @axapaxa in crypto, if one algorithm is newer, then that is a disadvantage and if all other things are equal, then the older algorithm should be preferred. If the strength is comparable and there are no currently known vulnerabilities, then the "obsolete" algorithm is more trustworthy and the new algorithm is more likely to have new vulnerabilities discovered because it's not been reviewed as much. – Peteris Oct 29 '16 at 9:17
  • 16
    @axapaxa Cryptographic algorithms are never abandoned because of age. They are only abandoned when evidence of weaknesses are found. The older an algorithm gets without any weaknesses being found, the more reason there is to keep using it - until the point where hardware gets efficient enough to brute force it. The reasons to stop using 3DES is that it is not particular fast and the block size is only 64 bits, which is far too small for most modern usage cases. A too new algorithm should OTOH only be used with lots of caution. – kasperd Oct 29 '16 at 22:21
22

MD5 and SHA-1 are no longer safe not because "most password hashes are now on the internet." The use of salts to make password hashes globally unique in fact makes this impossible. They are obsolete because they are too fast, and too many candidate passwords can be tested against a stolen hash too quickly for comfort.

The downside of hashing is that it isn't reversible. This is why it can't be used for all sensitive data. You bank balance, for instance, isn't much good to either you or the bank if it is hashes, and neither of you knows what it is.

  • 3
    Just to put some figures out there: in 2009, a high-end commodity GPU could calculate around 200 million MD5 hashes per second. Today, you can get a GPU 6 times faster than that GPU for under £100, netting you well over 10^9 hashes per second. Figures for SHA-1 are likely to be a little lower, maybe 1/3 or a 1/4 as much. If you have cash to burn on the project, you can get a single machine that can hash over 6x10^10 SHA-1 hashes per second, and that was 4 years ago. – Periata Breatta Oct 29 '16 at 15:39
  • How about a really slow invertible function for other data? Say, decryption takes of order 1s. Fine for a single lookup of my details, but painful for browsing everything. Is that ever useful? – innisfree Oct 31 '16 at 13:09
  • @innisfree I'd say you're thinking along the right lines, at least for some systems. Generally systems fall into one of two models. In the 1st model, someone (owners, administrators, etc.) need access to aggregate data, in which case this sort of function would be prohibitively expensive. – Xander Oct 31 '16 at 13:36
  • @innisfree In the 2nd model, where only data owners need access to their data, or data is never aggregated across multiple users it could indeed work, but we have an alternative mechanism that we already use which is even better....A PBKDF (like PBKDF2, bcrypt, scrypt, etc.) is used to derive a key, and that process is where the work factor you suggest comes into play. – Xander Oct 31 '16 at 13:36
  • @innisfree The key is then used (generally indirectly) to decrypt the data. This has the added benefit in that not only is it slow find the correct key, the correct key is only correct for one user's data. It not only makes brute-forcing the data for multiple users slow, but impossible. – Xander Oct 31 '16 at 13:36
8
  1. While it's probably not that easy with well configured systems in mind, multiple vulnerabilities either on the OS or application level (like SQL Injection, possibly even a file inclusion) may lead to database compromise, as is quite often the case in reality. Protecting passwords by hashing them is an example of defense in depth. Even if a line of defense fails, it should still be relatively hard to get to actual passwords.

  2. Passwords are a good target for multiple reasons. On the one hand, they allow impersonation of users, rather than reading, changing or deleting data with the attacker's (possibly compromised) account. Also users tend to reuse passwords, so a password stolen from one application has a very good chance to be valid for other accounts of the same user. As for why other data is not hashed - hashing is one way, you cannot get back the original value from a hash (at least not trivially, in reality and with many hash functions, you have a good chance, but let's disregard that for a moment). Being one way means that hash functions are good for checking whether a password entered is the same as the stored one, but it is not good for storing data that you actually need in its unencrypted form. And that is the solution you may have been looking for, encryption, as opposed to hashing. It is indeed a good idea to encrypt sensitive data in a database, but that too has its own problems, for example key management.

  • 3
    The username, for example, is not hashed because that is likely how the application finds which password hash it needs to compare -- it queries "return the hash from the row where user= entered-user-name " and hashes the entered password and compares that hash with the returned hash. – Doktor J Oct 29 '16 at 1:57
  • 1
    @DoktorJ Not to mention that usernames used for logins must be unique. Hashes don't have a unique-ness guarantee. That may seem obvious, but plenty of interview candidates have convinced me otherwise - apparently, a four byte hash of a random hundred byte sequence will always be unique :/ I wonder why we don't use such hashing functions as compression algorithms instead :D – Luaan Oct 31 '16 at 10:54
8

Is hacking a database that easy? I mean instead of finding new ways to hash, why can't they make their database more secure or "hack-proof"?

Often a database can be hacked through a web application using SQL injection. We absolutely should fix SQL injection as a priority. However, in a large application, it only takes a developer to make a mistake on one line to introduce such a vulnerability. So while we try to stop it, we also plan other defences in case a vulnerability gets through.

If a hacker manages to hack into some database, why would he want to know the hashed passwords? I mean he can change other columns of data. For example, he could search for his username and increase his bank balance. So, shouldn't all the fields be hashed?

In many cases SQL injection allows attackers to read data, but not change it. If passwords were not hashed, they could login as other users, and make changes. Password hashing helps prevent this. Also, users often re-use passwords on many web sites, even though it is bad practice to do so.

Why are only passwords hashed?

In a well designed application, all authentication tokens are hashed. This includes session IDs and passwords reset tokens, as well as passwords. Other data (e.g. your bank balance) is not hashed, because the application needs the data in plaintext format to operate.

  • 1
    Defence in depth is a key concept indeed. Also, you could mention hashing prevents legit users (such as DBA) from knowing the password. – spectras Oct 30 '16 at 13:02
  • @spectras - I tend to avoid that claim. Hashes do stop DBAs in some circumstances, but a malicious sysadmin could modify the system to log plaintext passwords. – paj28 Oct 30 '16 at 18:10
5

Is hacking a database that easy? I mean instead of finding new ways to hash, why can't they make their database more secure or "hack-proof"?

Why do we need hospitals? Why can't we make the road more secure instead of having hospitals?

A totally "hack-proof" system is an illusion. There are millions of ways to hack a system. You need to defend against all of them. An attacker only needs to find one.

This requires you to know about all ways. And that you didn't make a mistake (just look at the default setting for Mongo-DB: non-authenticated admin access and then combine it with many admins setting the database to be reachable from the internet for convenience or because of poor manuals). And have the resources. And to stay up to date when new flaws are discovered. And the same for all the providers of the software and services and hardware you are using. And that none of them had bad intentions. And that there are updates for new discovered flaws (Android smartphones are known to be not very well serviced with security updates). And that there are no unknown flaws that the attacker knows of.

Are you sure that you defended against all of them? Whom do you trust? How about the admin that was let go and still has access to a copy of the database backup? Did you erase the storage correctly before reselling or throwing it away? Who has access to the backups?

If a hacker manages to hack into some database, why would he want to know the hashed passwords? I mean he can change other columns of data. For example, he could search for his username and increase his bank balance.

  1. It could be that he has only read-only access to the database. For example when he got a backup in his hands when it was stored on an unsecured webserver or his attack vector only allows reading.
  2. He might only have access to one part of the system where the passwords are stored but not to the system where the balances are stored.
  3. Many professional databases have an audit trail. Let's say you change your balance in the database. That might let their alarms off because the system sees no corresponding reduce in another balance and no authorization for this transaction etc. and it is easy visible. If you instead log in as another user and press transfer 1000$ to another account it will look more legit.
  4. You can use the login data to impersonate another user and he will get the blame (many bigger provider already have heuristics to detect such things automatically or when claimed)
  5. Time of attack (-vector) and time of access: You only need to attack once but can use the credentials later (many users don't change passwords for years or never)
  6. Many less-technical users reuse the passwords on other sites
  7. You have more time until the attacker can impersonate the users after he attacked

That were the first reasons I thought of and there are many more.

5

shouldn't all the fields be hashed?

It's an interesting question, so let's consider the ramifications if (say) the username was hashed as well. Right now I am "Nick Gammon" on Stack Exchange. If my username was hashed then instead of a post by Nick Gammon, we would see a post by 80a8694f4a2950e075d142e97ddb809b415bf85f084dafd9fb78fed9551905f3 in response to a question by 76d6d4c28518362c96d55f52cfdf27908baadf6f37973eb42cfd515b4c96294a1. You can see this would be incredibly tedious. (Remember, you can't reverse a hash to get the original back).

OK, you might say "oh well, keep a login name (which you hash) and also a display name". But then if you saw my display name was "Nick Gammon" you might guess that was also my login name (or some simple variant of it), so hashing it hasn't achieved anything.

Then you might worry that if someone got at the database they could also see the email addresses, so you hash those as well. But you can't send an email to a hashed email address (eg. to notify you of new posts, or a dip in you bank balance) so that wouldn't work. OK, so you keep the email address as plain text, but now you can probably deduce the user name from the email address.


why can't they make their database more secure or "hack-proof"?

No design will cater for a trusted employee being bribed to make a copy of the database. If the data is there, then people with enough clearance will have to be able to get at it, or it may as well not exist.

I think some of the recent security leaks have happened to organizations that thought their database was so "hack proof" they didn't need to bother hashing their passwords. Oops.


  1. Extra marks for working out who that is. :)
  • 3
    Extra marks for working out who that is That's a trick question since the hash you provided isn't for "Nick Gammon", but is for "Nick Gammon\n"... After realizing that, then it was easy to figure out who that hash belongs to :) - it only took 2 guesses. Another nice thing about hashes is it lets me show that I know who it is without revealing the answer by giving you a hash of his name reversed (no newline): c4354c3d27541e605c2cab2a661ef17fb7fb3d404d61f71a903f9a119a1bf0a4 – Johnny Oct 31 '16 at 5:46
  • 1
    Ah, I see the problem. I did echo 'Nick Gammon' | sha256sum where I should have added the -n option. You are indeed correct as you confirmed. :) – Nick Gammon Oct 31 '16 at 5:53
  • The next person who wants to confirm might do an all upper-case version, or an all lower-case version of the name. However by saying that I am leaking some information. ;) – Nick Gammon Oct 31 '16 at 5:55
2

"Defense in depth" is the principle that multiple, redundant layers of security are best, and more specifically that no component of a secure system should trust any other component of a secure system. In this case, the hashing layer should not assume the database itself is secure.

You asked other technical questions about hashing that are covered in detail in the other answers, but the more fundamental principle of defense in depth - "why lock your door twice?" - is something you seem to be missing.

I'm sure you heard enough of high-profile cases of security breaches in 2016 to know that, in aggregate, we are doing far too little, not too much.

2

Do not restrict yourself by only analyzing the attack from outside. You have to see the bigger picture... think of security measures as layers of an onion. You stack as many as you can in the hope to close off any angle of attack. Password hashes are one of those layers.

Is hacking a database that easy? I mean instead of finding new ways to hash, why can't they make their database more secure or "hack-proof"?

It does not matter whatsoever how hard it is to hack the database. Consider that password hashing is supposed to protect the password from even the system or DB administrator.

Note that even if you would try to do so, it is not necessarily easy to protect the password database. How would you protect it? With another password? Where would you store that master password? How would you protect the master password against someone being able to read everything on the system?

If a hacker manages to hack into some database, why would he want to know the hashed passwords?

Well, to try and attack them.

Note how passwords are used: the legitimate user hands a username and a cleartext password to your login screen, the cleartext is then hashed and compared to the password hash.

Every attacker can do exactly the same, except doing it over the actual login screen/prompt is inefficient - it takes time and is easily detected; many systems will lock an account after a few bad attempts.

So when the attacker gets the list of password hashes, he can do the same hashing in his own program - blindingly fast, and especially with "dicitonary" or "rainbow" attacks, which trade RAM/storage against time.

So, shouldn't all the fields be hashed?

You cannot hash all fields because your application needs to know the value of the other fields, simply. It is hard to get the value back from a hashed version, even for a legitimate user.

But in fact, there are databases which do something like that - they encrypt (not hash) the actual data as well. You want to weigh whether it is worth it from a performance point of view, and you then have to make sure the encryption actually provides any real security. I.e., you have to make sure your key stays protected while the DB needs to have access to it, and so on.

Encryption is not appropriate for passwords because we actually do not want the passwords to be able to be decrypted (same ideas as at the beginning => inside jobs, and the difficulty to make the encryption key more secure than the actual passwords you wanted to secure in the first place).

2

Because it is not just remote, unauthorised access that has to be protected against. Sometimes whole data centres go missing. Once the bad guys have the disks that hold the database files they can be read at leisure. Then all the unencrypted data can be taken - credit card numbers, names, address, email, phone numbers, purchase history. Anything and everything that facilitates money theft or identity theft.

Current good practice is to encrypt anything that is personally identifiable or financially sensitive. Several DBMSs support encryption at rest (i.e. on the disk) and in-flight (while being transferred from DB server to application).

0

Is hacking a database that easy?

No it is not that easy. A poorly configured setup is what makes the databases vulnerable and expose it to attackers. One should apply any patches available for the version of the database that is used, and if possible one run the latest version.

I mean instead of finding new ways to hash, why can't they make their database more secure or hack-proof?

People keep trying to find new algorithms for hashing because collisions are detected in the current hashing algorithm. It is not like we are unable to find ways to secure databases.

why Hashing?

  1. Hashing ensures the end user that their secret tokens and passwords are stored in an irreversible format.

  2. Hashing ensures that the application which is accessing the database do not get access to secret data in clear text form.

  3. Hashing also prevent secret data from getting exposed even to database and system administrators.

If a hacker manages to hack into some database, why would he want to know the hashed passwords? I mean he can change other columns of data. For example, he could search for his user name and increase his bank balance. So, shouldn't all the fields be hashed?

Defacing the hidden database through the applications having access to it is different from breaking into the databases server. If an attacker breaches into the server hosting the database then of course he can do anything allowed for whatever privilege level he got access to.

But don't you think getting passwords in clear-text is dangerous then in a form that is irreversible, because the only possibility left for the attacker is to launch passive brute-force to find the passwords, which gives enough time for the victims to change their compromised passwords.

You can very well see the difference in strength of cryptographic challenges that the attacker will face to reach this hash.

Why are only passwords hashed?

Passwords authentication is used to prove authenticity of an entity, so possession of a password is enough for an attacker to be able to impersonate another user. So there is a security risk of identity theft if not hashed.

-2

There some solutions that encrypts all sensitive data in your database, for example Azure. But the security is not free and cost money, time and you can get worse performance because of encryption. No lunch is for free.

Imagine some one-man e-shop business with low budget. This business cannot afford some expensive solution. Their choice is to choose the cheapest one or nothing. Security is not a choice, because you don't have money for it. Security is starting to be important as company grows.

  • 1
    The question talks about hashes, but your answer is about encryption. – Anders Oct 30 '16 at 19:32

Not the answer you're looking for? Browse other questions tagged or ask your own question.