11

Assuming I have secret data that is encrypted (using sops for example) and a checksum of the secret data for change detection: Is it possible to derive secret information from the checksum or should the checksum also be encrypted?

I would assume, that no secret data can be reconstructed just by having the checksum.

For the sake of this discussion, let's assume we have a Kubernetes secret YAML sops encrypted and the checksum of the unencrypted YAML file is generated using sha256sum.

Deriving from the comments, it might be important to point out that there are some "fixtures" in the Kubernetes secret manifest YAML that are well-known:

apiVersion: v1
kind: Secret
metadata:
  name: <dynamic secret name>
type: Opaque
stringData:
  <dynamic content>
6
  • 4
    It depends not only on the checksum algorithm, but also on the data: What is known about the data nature and how big the data set is. For instance, if it is known that the data set (message) consists of 2-3 words out of known dictionary of 5000 - 10000 words, then brute-forcing may well be successful. If this is not the case, then it will be helpful, if you add such details to your question.
    – mentallurg
    Mar 6, 2023 at 21:54
  • @mentallurg to be fair, bruteforcing isn't about deriving the secret from the checksum itself. In bruteforce, the checksum isn't primary.
    – schroeder
    Mar 6, 2023 at 22:07
  • @schroeder: Brute-forcing is not about what one searches, but about how one searches. Brute-forcing is an approach where one generates and tests all possible values, trying to find those that fit particular conditions. The purpose of the values searched does not matter. In particular, testing all possible values from some set trying to find a preimage is of course brute-forcing.
    – mentallurg
    Mar 7, 2023 at 1:09
  • @mentallurg I understand all that. What does that have to do with "deriving the secret from the checksum"? Brute force is parallel to the checksum.
    – schroeder
    Mar 7, 2023 at 6:43
  • 2
    @schroeder: " Is it possible to derive secret information from the checksum" - I believe that derive is wrong word here (the author is not a native English speaker and is living in Germany) and that instead restore was meant. The author is asking if the secret information can be restored from the checksum. And if the amount of unknown information is relatively small, it obviously can be brute-forced.
    – mentallurg
    Mar 7, 2023 at 8:28

5 Answers 5

17

sha256sum is a cryptographically secure algorithm, compliant with FIPS-180. Such algorithms are designed so that one cannot derive the secret from the hash/output.

Other methods can be used to try to guess the secrets, like bruteforcing, but that's not a weakness of the checksum.

1
  • 7
    The only deriveable information would be that an attacker that gets holst of just some random data could confirm that what they got is indeed the secret data. Though given OPs statistics that would likely be obvious anyway
    – Hobbamok
    Mar 7, 2023 at 8:16
13

Is it possible to derive secret information from the checksum?

Yes. The checksum will disclose (at minimum) a Yes/No on whether the secret is a given value (i.e. act as an oravle).

If you want no information leak at all, you should either:

  • hash/checksum the encrypted secret
  • encrypt the checksum (either along the secret or separately)

Note that this is generally not a big issue if

  • the checksum function is a cryptographic hash, like your usage of sha256, and
  • the checksummed data contain enough random information unknown to an attacker (either because that is included in the secret, or there is a separate source of entropy included in the checksummed data, which is why adding a fresh UUID4 each time was suggested at another answer)

but the other options are better ones.

Note that even with cryptographic hashes, an oracle can be a powerful primitive. For example, if an attacked knew that some time ago the secret was:

kind: Secret
stringData: StackExchangeFeb2023

knowledge that the current hash of the secret is 15c42421218cd40e6051f20c0993604ffa6583b746a068867ec864d9c5912fc1 would allow them to successfully guess the new value, despite the preimage resistance of the hash algorithm used.

4
  • 1
    Indeed, this is basically the problem of encrypt-then-MAC vs. encrypt-and-MAC vs. MAC-then-encrypt: en.wikipedia.org/wiki/…
    – Nayuki
    Mar 8, 2023 at 4:57
  • 1
    @Nayuki this is a bit different, since OP's checksum is public and could be easily forged (not sure if he's aware of that), but they are certainly related problems
    – Ángel
    Mar 8, 2023 at 20:06
  • 1
    +1, though I'd also suggest a third option, which is to use an HMAC rather than a simple hash. That gives strong resistance against forging checksums; and it can be combined with your suggestion to hash/checksum the encrypted secret, with the benefit that the checksum can be checked without having to decrypt the secret.
    – ruakh
    Mar 9, 2023 at 8:21
  • It does also allow you to tell if a value has changed even if you know nothing about the data contained within. This may or may not be relevant depending on the specific use case and can usually be easily mitigated by adding some random junk data that will be discarded. Mar 9, 2023 at 20:52
6

If you put something like a human generated password into the secret data and an attacker gets to see the sha256, you have basically used sha256 as a password storage hash, which is very weak - the attacker can brute force it offline using multiple GPUs at many billions of guesses per second (a single RTX 4090 does over 21 billion hashes per second for sha256), using known password files, and they'll break many human generated passwords.

If the secret data has a lot of entropy - meaning it has computer generated long token, at least something as complex as a fresh UUID4 generated a moment before the value is hashed and encrypted, then it's safe.

6
  • 1
    could you elaborate why the freshness of the UUID4 plays a role here?
    – mxcd
    Mar 7, 2023 at 9:33
  • 4
    @mxcd - it needs to be a UUID4 the attacker doesn't know. Provided the attacker doesn't know it, it's fine - but a fresh one is less likely to have leaked/been reused. Mar 7, 2023 at 9:51
  • 3
    You're mixing up concepts here. When you do password lookup on the hash, that's not considered "brute-force". It's just a table lookup. Strictly speaking, you can't brute-force SHA256 due to the sheer quantity of combinations, 1e77. The GPU is only 21e9, and a billion GPUs (1e9) running for a billion seconds (1e9 sec = 32.5 years) is still only 21e27, off by almost 50 zeroes.
    – Nelson
    Mar 8, 2023 at 1:34
  • 2
    @Nelson you don't need to bruteforce the entire space of sha256, you brute force the input space. If my secret data is in the format "<StockTicker>: <BuyOrSell>", its easy to determine the data from the hash by bruteforcing all possible combinations.
    – mbrig
    Mar 8, 2023 at 21:27
  • @mbrig I don't think your scenario is what OP is referring to. They're hashing the entire block of YML data and using it to detect changes, so the input space is arbitrary and effectively infinite. I'm only addressing the statement "attacker can brute force it offline using multiple GPUs at many billions of guesses per second" which is wrong and can't be done on SHA256. Your scenario is brute-forcing the input space, which isn't the same as brute-forcing SHA256.
    – Nelson
    Mar 9, 2023 at 0:36
2

Insufficient Complexity

Let's pretend you have a secret password, "bad".

You calculate the SHA256 of this password and get "2f05d4b689d270cafb02285f35f44866f7dc8a2d368a3f9d1124373eeab31fb1".

If you ALSO tell me that it's the hash of a password that is less than 4 lowercase characters, I can obtain your password in seconds by brute forcing. I only need to try 26^4 combinations for a total of 456976 sha256 hashes to exhaust all possibilities.

Give it a try yourself: https://ideone.com/hSWQ8R

from hashlib import sha256
import string
 
secret_hash = "2f05d4b689d270cafb02285f35f44866f7dc8a2d368a3f9d1124373eeab31fb1"
 
def pass_generator(max_length, charset):
    for i in range(len(charset)**max_length):
        yield get_pass(i, charset)
 
def get_pass(num, charset):
    password = ""
    while num > len(charset) - 1:
        num, index = divmod(num, len(charset))
        password += charset[index]
    password += charset[num]
    return password
 
for password in pass_generator(4, string.ascii_lowercase):
    if sha256(password.encode()).hexdigest() == secret_hash:
        print(password)

Common Passwords

Let's suppose you instead have the secret password, "123456".

If I do a quick google search and get a list of the 10,000 most common passwords, I can search fewer, potentially more complex passwords and in this case yours is definitely in there.

Common Patterns on Dictionary Words

Ok, ok, maybe you use "Sm@11Fry!"

Well, crackers have tools to generate and test possible passwords based on patterns, with common letter/symbol/number substitutions and suffixes. It would definitely take longer than the last two, but it's still possible that a generator would come up with this based on manipulating dictionary words with masks.

Insufficient Complexity, Space-Time Tradeoff Version

Ok, so how about "a#g%6^NN!"

Suppose that over many, many years I've used brute force to hash many, many different password-sized strings, including enough symbols to cover what you have here. Now suppose I find a data structure that would let me compactly save the relationship between the hash and the password. So now if I can look up your hash in this data structure, I'm essentially getting several years of brute forcing almost for free (yes, someone had to do the hashing originally). This is called a rainbow table and it's a fascinating concept I encourage you to read up on.

To limit the effectiveness of rainbow tables, you can salt your secret: generate some random bytes, and append them to your password before hashing.

Now my existing rainbow tables are useless - I'd have to recalculate them from scratch for your exact salt (assuming I could obtain it). This is an even bigger deal when you recognize that salts are typically per-user in a real authentication scheme, so an attacker who obtains the whole table of hashed user creds and passwords and salts doesn't get any reusable value from brute forcing the password space for just one of the salts.

Your Case

Suppose I had a yaml like so, and I hash it. Later I use sops to encrypt the secret string "bad", and the hash and the sopsified yaml are available to the cracker:

apiVersion: v1
kind: Secret
metadata:
  name: my-cool-secret
type: Opaque
stringData:
  bad

I could brute force this just as easily as the password example above. I assume I would know the exact structure of the yaml, so the only variable part is the stringData, and if I expect it to be all lower case and very short, it's the same complexity of brute force.

Try it for yourself: https://ideone.com/P0FNg6

from hashlib import sha256
import string
 
template = """apiVersion: v1
kind: Secret
metadata:
  name: my-cool-secret
type: Opaque
stringData:
  {}"""
 
secret_hash = "2f97f7393a589fff32db98e5edf54a455937516f567b0d352556537fcb06aa53"
 
def pass_generator(max_length, charset):
    for i in range(len(charset)**max_length):
        yield get_pass(i, charset)
 
def get_pass(num, charset):
    password = ""
    while num > len(charset) - 1:
        num, index = divmod(num, len(charset))
        password += charset[index]
    password += charset[num]
    return password
 
for password in pass_generator(4, string.ascii_lowercase):
    if sha256(template.format(password).encode()).hexdigest() == secret_hash:
        print(password)

All of the same cracking principles apply as with the simpler examples above, except that multiple secrets extend the total complexity.

1

The big keyword here is that I will/could know your hashed secret. If I know what your hash is, I can figure out what the secret value is NOT by hashing something else with the same algorithm. If the maximum number of possibilities for your secret value is less than the number of possibilities I can hash, I can in theory, determine your secret value. If your secret value is something like an IPV4 address, which has at most ~4 billion possibilities, using modern hardware, I could hash 4 billion values very quickly, and utilizing something called a rainbow table, figure out your secret value very quickly. However, if your secret value is something that has a very large, or theoretically infinite number of possibilities, then it would take a very large, or theoretically infinite amount of time to figure out what it is.

Think about it like this. Your secret value is fixed. Meaning the probability of me guessing at random your secret value is: p = 1/n; where n is the size of all possible values. Therefore, if I wanted to guarantee I guess your secret value, all I would have to do is guess all possible values. success = t(h) * n; where t() is the amount of time required to compute a hash (h) using your algorithm.

If n is small, say 4 billion and if t(h) is also small, say 10mhs, or 1/10E6, then you could guess all possible values in about 400ish seconds. However, I am sure you can see how as we start to raise the value of n, the amount of time required also increases. Alternatively, you can use a more computationally complex algorithm to increase the amount of time required for me to compute each hash. Doubling the computational time required for each hash will have a small effect on you, but it will literally double the amount of time required to guess your secret value.

So the question becomes, how many possible values are you able to put into a Kubernetes secret file?

1
  • Welcome to the community. In this case it seems to rather a password and not an IP address, so it should be alphanumeric in my humble opinion Mar 7, 2023 at 20:30

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .