We can calculate the Entropy S of a string simply like this:

S = L * log_2(N)

Where L stands for used characters in the string

And N for every possible character

Example: We want to calculate the Entropy for a PIN with 4 digits.

Our N would be 10. Because we have 10 numbers (0-9). Our L is 4 of course.

So we have a Entropy of about 13.28 Bit (or 3.32 Bit per character)

But how do we calculate the Entropy for let say an ’empty string’ by using e.g. md5sum, sha256sum

And most importantly: What is 𝑁 and what is 𝐿 here?

  • I'm very unsure of what you're asking here... Are you asking about the hash entropy ? – Tensibai Jul 2 '18 at 9:09
  • First, you simple "perfect" entropy applies if the data is perfectly random - not in any other case. – user155462 Jul 2 '18 at 9:11
  • 2
    Second, where suddenly the hashes come from? And an empty string has L=0 according to your own defintion, so the result is 0 too. – user155462 Jul 2 '18 at 9:11
  • @Tensibai yup. Sorry if my question was too unclear – Azrion Jul 2 '18 at 9:15
  • @user155462 Youre right but when i generate a MD5 hash with just no input value (=empty string) the hash would be d41d8cd98f00b204e9800998ecf8427e. Now the question is: do we still have a Entropy of 0 here? – Azrion Jul 2 '18 at 9:17

TL;DR: Your formula is too simple.

So the problem is why "" has an entropy if 0 with your formula, but MD5("") has not 0.

As said already in the comments, your formula only works if the data is perfectly random (uniform distribution, all bits independent, etc.).
Example for random case: Your PIN calculation with 4 digits between 0 and 9.
But if you have eg. some value that can either be 2222 or 7777 and nothing else, the entropy is just 1 bit. There are still 4 digits, but most 4-digit numbers are not possible, so your formula doesn't work anymore.

The same is true for your hash - hashing a 0-length string gives exactly one output, always the same. Not 256^16 possibilities that 16-byte hashes can have.

A better formula:

H(X) = sum (-p[i] * log2 p[i])

where p[i] is the probability of the i-th possible value (not digit etc., but value - your PIN has 9999 possible values, and each one has its own probability).

With this formula and the one single possible hash value, you get 0 too.

In general, a hash like this never can increase entropy. It may be the same or less than the one of the original value (less especially if the data is longer than the hash, ie. the hash has less possible values)

  • very good explanation. thank you user155462 and Tensibai! – Azrion Jul 2 '18 at 9:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.