3

I want to do an anonymization for user data in my simple system. Each user have two files, for example <id-number>-1.json and <id-number>-2.json.

I want to store them in some repository of all users files (the repository is secured). I want to move the files to this repository and anonymize the user ID number (without the option to receive the ID by the file name), and I want to make the new file names with different hash function so no one will be able to understand that these two files are connected to one user.

The goals:

  • To be able to access both files by the user ID.
  • Other people seeing even one file name will not be able to identify the owner user.
  • Other people seeing both file names will not be able to know that they are owns by the same user.

I thought about some solutions for this situation:

  1. First file: SHA512(<id-number><40-characters-suffix1>), Second File: SHA512(<id-number><40-characters-suffix2>)
  2. First file: multiple times AES(<constant-long-string>, <id-number-as-key>), Second File: multiple times AES(<other-constant-long-string>, <id-number-as-key>)

Which way is better to use? There are another better way to do that?

  • Can you explain the problem in a more generic way? Do you need to be able to find the files using a user id, without other people seeing the file names being able to identify they are from the same user? – kmaork Jan 7 '18 at 15:09
  • @Ni. True, exactly what you said and without other people seeing even one file name being able to identify the user that owns it – nrofis Jan 7 '18 at 15:27
  • How about saving the filenames as md5(secret_key+original_filename)? – kmaork Jan 7 '18 at 17:25
  • @Ni. it is the same idea as my first solution because the original file name is the user ID – nrofis Jan 8 '18 at 13:40
  • But if you concatenate the user name with an incrementing number, the filenames will be different and brute-force won't work because of the secret key – kmaork Jan 8 '18 at 19:54
2

I'd go with the first system, there's no significant difference from the AES one; but still you need to consider other, possibly unrelated factors.

UPDATE: on second thoughts, you might want to use PKDBF2 as a "hash", in order to mitigate the chances of a successful bruteforcing.

Brute force attack

If the suffixes are known, or in general there is a known mapping between user ID and hash, then an attacker could simply enumerate all possible IDs.

For example, the attacker knows that IDs start at 1 and are at most 100,000; even using a CPU- and memory- hard algorithm, generating the hashes for all possible user-IDs and storing them into a database wouldn't take more than a few days. Using SHA1 or AES, it's a matter of minutes, perhaps even seconds. Afterwards, to know what ID hides behind a given hash, it's just a matter of looking for that hash in the list.

You might want to consider replacing sequential IDs with random ones (which can be a pain in the nether regions; and you need to handle the case where a random newId collides with a yet-to-be-replaced oldId). At that point the "possible IDs" are no longer one hundred thousand (from 1 to 100,000) but several billion trillions (from 0x00000000000000000000000000000000 to 0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF). Bruteforcing will now take an impossible amount of time, and/or an impossibly large database to hold the hashes.

Correlations

Consider the possibility of non-trivial correlations between file1 and file2. While a trivial correlation would be both files containing a unique identifier, or worse still, a copy of the user ID, there might be information in both files that could allow determining the user ID, or the link between paired files; for example, a timestamp, or a invertible UUID for some resource.

This issue might extend to metadata such as mtime, or inode number, or position in the directory listing; two files created within the same second, or stored one after the other, could be tracked to the same user, even if one can't backcalculate the user ID.

You should consider archiving the files in a 2- or 3-deep structure, i.e.

/rootdir/ae/02/47/ae0247f19a8b52f.json

and using a non-atime-updating filesystem; and/or when you do change or create or delete a file, touch the whole branch (here ae, ae/02, ae/02/47 and ae/02/47/ae0247f19a8b52f.json) to a fixed datetime, such as midnight January 1st, 1970.

Initialising the repo

When transferring the files to the repository, you might want to first produce a list:

 000001-1.json      aae3799a82b9fb8ccd34c1d5aa6565b2
 000001-2.json      539c50984894084b3e3b1047eee187ae
 ...

Then you scramble the list. Once done this, for each pair on the list you move e.g. 000001-2.json to repositoryRoot/53/9c/50/539c50984894084b3e3b1047eee187ae. This way, there will be no correlation between the inodes, filesystem position or disk physical position of 000001-1.json and 000001-2.json.

Be warned that on most filesystems, this approach will appreciably increase the time required to do the copying.

Emails as IDs

This does not look very promising security-wise, because emails are often guessable and/or recycled. Much would depend on what exactly is stored on those JSON files and what the attack scenario actually is (the two cases of a hacker gaining access to a repository of embarrassing information and attempting to blackmail the owners - thus requiring to work back from the JSON to the email - and someone trying to acquire the information of a victim whose email is known, are to be handled very differently).

One possibility could be having an anonymisation table kept outside the repository:

ID                   RandomToken
lserni@gmail.com     5231b225ea0dbcced14c993523af4986
....                 ....

At that point you could use token as the "secret" password.

|improve this answer|||||
  • If the user ID is his email address and the suffixes are private? The broth-force attacks are still a problem? BTW, thanks for your detailed answer, you gave me another perspective :) – nrofis Jan 7 '18 at 16:32
  • If the suffixes are private the bruteforcing gets well-nigh impossible, but you shouldn't rely on them remaining private; should the prefix leak, it would be a merry hell to get things fixed. I've updated the answer and changed my suggestion. – LSerni Jan 7 '18 at 17:36
  • Just for be sure, if I will user PKDBF2 hash, the "password" is the user ID (email address) and the "salt" is one of my constant suffixes? – nrofis Jan 8 '18 at 13:39
  • Yes, provided that the emails are unknown and unguessable by the attacker. Otherwise you'd need a second secret, and/or a email/randomID table kept outside the repository. – LSerni Jan 8 '18 at 14:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.