MD5 before PBKDF2 on legacy software

Question

I am mantaining a legacy software that uses MD5 in the following way:

1. The client asks for a 16 byte salt when authenticating an account.
2. The server generates a 16 byte salt, links it to the account and sends it back to the client.
3. The client appends the 16 byte salt on a password, does MD5(pass + salt) and sends the MD5 hash through the network.
4. As the server stores the plaintext password in the database, it only retrieves the password and compares MD5(database_pass + salt) with the received MD5(pass + salt) from the client.

As I can't change the workings of the client, I was thinking about what I could do to stop storing the plaintext password on the database. The idea I came with was to, instead of storing the plaintext password on the database, to store the 16 byte salt used by the client and, at the registration, generate the MD5(pass + salt) and use this MD5 as the input to PBKDF2 with a 64 bytes salt and HMAC-SHA512, 10000 iterations. Then, to authenticate, I would receive the MD5(pass + salt) from the client and PBKDF2 it to check with the stored PBKDF2 hash.

My doubt is, will this make my server more secure? As the password and the MD5(pass + salt) won't be stored on the database, the password won't be accessible anymore if my database leaks, but is there a problem with using MD5 before?

Are there any possible problems with this approach? Is there a better approach to it?

The only drawback I can think of is that I won't have access to the client's password after a successful authentication, so changing the authentication system later will be difficult. I will have the MD5(pass + salt), but then changing the salt would be impossible.

I want to do this right because I won't be able to change the system later. Any help is appreciated.

Answer 1

I think there is a serious flaw in your suggestion: unless I miss my mark, your "salt" isn't a salt at all: it's a challenge and, as such, it should be randomly generated each time a client attempts to authenticate.

If this is the case, then you cannot use it as input for the pbkdf2 function. It will just not work. If it is not the case, then unless you're already using a secure connection between the client and server, your auth scheme is seriously flawed: it is vulnerable to a replay attack and sniffing just as much as a plaintext password.

Overall, it's always better not to try to implement these kind of scheme: use a secure connection (TLS) between the client and server to protect data in motion and use PBKDF2 (or any standard key-derivation function that matches your requirement: BCRYPT, SCRYPT, etc.) to protect data at rest. If you really need challenge-response, then use SCRAM.

In your case, unfortunately, implementing SCRAM would require a change in the way the client works.

The best alternative would be for you to store the password in (reversibly) encrypted form. However, protecting the keys properly a very complex task: difficult to get right and easy to get wrong. It's hard to make suggestion for this type of implementation since it depends much on the value of the assets you're protecting (I would suggest, however, that you consider moving the authentication to a completely different system that only interact with the main one through a strict interface)