In a nutshell, I need:
- to hash a file and to make this hash public, to be able to prove data integrity when sharing the data privately with a trusted third party.
- to choose the right hashing method to prevent plausible brute force attacks by an attacker who know some important features of the data. I guess it involves using some salt/pepper methods, but I'm a newbie and not sure about the right method: hmac(data, key), sha256(concat(data+salt)), or something completely different?
Here's my detailed use case:
I have some survey data containing personal information, for example under the form of a csv file:
etc. (that's just an example, some surveys may have more simple structures, others more complex).
I cannot share the raw data publicly for privacy reasons, even if I can and want to share it privately with a handful of trusted third parties (let's call them party A).
I want to generate a hash of the data, and publish this hash on a third party public server (github or whatever) at time X. The goal of publishing the hash is to be able to prove later to party A that the data have not been manipulated or tampered with between time X and now.
I may not necessarily know who is party A at time X, so simply sharing the data or the hash privately with them at time X is not an option.
Also, I don't want an attacker to be able to "reverse engineer" the data, assuming they'll know how the data is structured -i.e. they may know all the existing column names in the csv file, as well as all the possible values inside each column. For example, they know for sure there's an "Age" column in the data, and that all the "age" values lie between 0 and 90 (but they don't know how old a specific person is in the dataset, they just know she's aged between 0 and 90).
It's my understanding that just hashing the data and making the hash public make the data susceptible to a brute force attack (the attacker could generate a set of plausible csv files, hash them, and compare all these generated hashes to the public hash until finding a match). As aggregate data will be made public (e.g. information like "90% of men in the data live in New York and have an income between 50,000$ and 150,000$"), I guess it would make a brute force attack much more easier.
From what I've read on the subject, I understand that I should use a "salt/pepper" approach, where I'd add a private salt to the data and generate the hash from that, and then privately share the salt with trusted party A.
But I'm not a security specialist at all, so I'm not sure which specific method is the best to do that. For example, I've read this discussion https://stackoverflow.com/questions/64325456/is-it-appropriate-to-use-an-hmac-and-pepper-to-hash-input-from-a-limited-domain, but I'm not sure at all if hmac would be actually appropriate for my use case. Many resources about salting seem to talk about passwords, but I'm a bit wary that the advice I've read does not apply to data files.
Any idea or suggestion? I'm a bit new to this kind of problem, so I'm very comfortable with hearing that I'm completely off track with this.
If it can somehow help answering my question, I work under Windows (job obligation), generally with Python, but I've no problem using other programming languages or tools if necessary.