I'm trying to convert a data field to something tokenized(that's to something not-sensitive). The current scenario is, I've got a data field (assume user-id) which is unique for each user, I don't want to store that, rather I want to run a function on it and then store the output value. All the user-ids are unique(no two users have the same id).

Now, I want the following properties:

  1. It should be secure, that is nobody should be able to derive the original user-id from the output. (This hints towards using a hash function, preferably SHA-2 or SHA-3 as they are more secure than MD-5 & SHA-1)

  2. The output should be the same each time for a user i.e. if I run the function hundreds time on a user-id, the output should remain the same.

  3. It should be computationally efficient.

To achieve the same, I'm thinking to use the following procedure:

  1. Compute the hash of the hash of user-id & salt concatenated with the salt value on both sides. It will help up to some extent in preventing birthday attack, and length extension attack. I'll be using sufficiently large salt to counter birthday attack.

enter image description here

  1. The hash output size of SHA-512, as the name suggests is, 512-bits or 64-bytes. I'll remove the first 4-bytes of the output and store the rest in the database. As the output is still long(60-bytes), the probability of collision will be almost negligible and for my current use-case, it doesn't matter if I have some collisions. This way, even if the database gets compromised, there is no way to know for sure that the given hash_output belongs to the particular user.

According to my use-case, it is computationally efficient. But is this procedure secure?

PS: In case this detail matters, all of this will be happening on the back-end. Once I get the data field, the information is never shown to the user again.


Please don't do weird constructions with the salt concatenated three times and the hash function run twice. It looks like you might be trying to invent HMAC badly, for no reason.

Even 128bit values can't collide in practice, you don't need 480 bits.

With per-user salt this is unsuitable for lookup, so it can't be a user-id. If it's not used for lookup but only like a password, please just use a normal recommended password storage method: argon2, scrypt, bcrypt, pbkdf2 in that order of preference, each one configured to the difficulty suitable for your application (i.e. number of rounds and amount of memory required for the memory-hard functions).

Look at hashcat benchmarks on a decent GPU and don't use a method that gets giga or mega hashes per second. An implementation of bcrypt or PBKDF2-HMAC-SHA512 should be available for any language you need, tune the iteration count and you'll be fine.

If it's not like a password, just a hash with 128 bit per-user salt would be fine.

  • I have a couple of things here to add: 1. It is not a password field, otherwise, I'd be using the standard method as recommended by you also. 2. These methods are computationally expensive and I don't have that much computational power as I've to process all this is Near-RealTime. 3. I was just reading about this 5 server 25GPU cluster which can scan 350 billion guesses per sec. If the salt is sufficiently long, I think this issue can be mitigated. Do you have a guide/best practice link on how what I'm trying to do is generally done in industry? – Divyanshu Apr 8 at 1:24
  • 1
    The salt doesn't slow down the GPU at all. It computes the giga hashes per second even though you have a long salt. The salt prevents stupid pre-compute attacks nobody uses anymore because everyone has long salt. There are really only two mitigations: either the real value being protected is itself high-entropy, like computer generated key and not like a user chosen password, say more than 80 bits of entropy, or the password hash is really slow and memory-hard. If both are false, the password will be cracked. – Z.T. Apr 8 at 1:32
  • Consider how "get token in exchange for password" solves the CPU cost of HTTP Basic Auth if password is stored securely (no need to check password on every request). Can you change your protocol to give a short term auth token in exchange for the password, so you won't need to authenticate using the password as often? The short term token can live in redis or something. (I call your value a password because you wish to protect it). – Z.T. Apr 8 at 1:35
  • Let me give you an overview of the design: The way my system is designed is that I collect data points continuously via some devices, each row in the data packet I collect has a field that uniquely identifies a user. That field doesn't have any meaning for me except to uniquely identifying the user. Now, I don't want to store that field directly but in some cryptic form so that it can not be misused by an adversary. But the cryptic value should always remain the same for that user. Due to the computational limit, I can't do this transformation on the device side. – Divyanshu Apr 8 at 1:54
  • If it's something low-entropy like a 32bit number, there's nothing you can do about the adversary being able to check all 32bit numbers. Would encrypting the value (AESGCM or ChaPoly) on the server using a per-server key and per-user nonce work for you? Without the key, a db dump would be useless, and it's fast. – Z.T. Apr 8 at 2:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.