7

This is something I never understood about hashing functions. I know that algorithms like whirlpool and blowfish both produce outputs that don't follow this pattern, but why is it that most do? Is it some kind of hardware/software thing? If they did produce outputs that went a-z0-9 instead of a-f0-9, wouldn't that increase their complexity?

  • 2
    Hash functions output a string of bits. The way you display them textually doesn't change their nature, only their encoding. – Thomas Oct 7 '12 at 0:51
20

It's just hex encoded. A 16 byte md5 hash can contain non-printable characters, so it's encoded to a 32 char hex string.

  • This is correct. MD5 is 128-bit (16 bytes). It's encoded in hexadecimal so the resulting string is easier to work with and debug. The actual hash is always in binary, but in most programming languages and libraries, the default output from a hash function is an ascii/utf8 string of hexadecimal encoded binary string. – Matrix Oct 6 '12 at 22:27
  • 1
    I would add one could use base64 encoding and end up with printable characters (this is more common in encryption embedded in text mediums, like pgp in emails, or otr over xmpp) – ewanm89 Oct 6 '12 at 23:28
  • 1
    You can use any encoding you like - raw bytes, hex, base64, base10 32-bit little-endian DWORDs, etc. – Polynomial Oct 7 '12 at 0:11
  • 1
    @Polynomial yeah, I was just pointing out that bae64 is also a common encoding to get printable characters and is used quite commonly in security, at the end of the day, you could write it out as a string of 1s and 0s.... – ewanm89 Oct 7 '12 at 0:41
6

As others have responded, hash functions (all of them, including MD5, SHA-256, Whirlpool and the dozens of other functions) output bits. The output of MD5 is 128 bits. However, humans are bad at reading bits. Humans are good at reading characters. So, when a hash function output is meant for human consumption, it is converted to characters with some encoding.

Hexadecimal is one of the simplest encoding of bits into characters. It converts a chunk of four bits into a digit or a character in the 'a' to 'f' range. Other encodings exist, e.g. Base64, which uses a bigger alphabet (64 signs instead of 16) and is more compact (the 128-bit output of MD5 is encoded as 24 characters in Base64, vs 32 in hexadecimal), but is a bit more complex to implement.

There are many possible variants with hexadecimal; e.g. we can add some spaces every few characters (to ease reading), or other punctuation such as colon signs (':'); we can use uppercase or lowercase letters. Tradition for hash function outputs is to use lowercase with no space or punctuation. This tradition was already in force in the infancy of MD5: MD5 has been designed in 1991, published in April 1992 as RFC 1321; as you can see in that document (near the end, section A.5), hash outputs already use lowercase hexadecimal.

1

Thomas Pornin's answer is great, but there is a further point. It is not just tradition and human readability in the output of a hash, there are places where it applies to the input. In RFC2069 on Digest Access Authentication, the hash output is re-hashed when concatenated with more data. But counterintuitively (to me) the representation of the hash used when re-hashing is the lower case human-readable hex character version rather than just the hash output as a bitstream, I.e the input is twice as long as it needs be. The RFC in section 3.1.3 specifies this representation 'for the purposes of this document', which is not as unambiguous as 'for the purposes of this algorithm' might have been, but my testing shows it to be the case for Firefox's implementation with MD5 and clearly all implementations must match. Conversely Bitcoin's double SHA-256 hash uses the binary representation How do you perform double SHA-256 encoding?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.