How do I select the correct hash value / Why is my hash so extremely long?

Question

I have used zip2john to get the hash for a zip file like that:

zip2john.exe myfile.zip > hash.txt

john told me:

ver 2.0 myfile.zip/SomeDir/SomeFile.itv PKZIP Encr: cmplen=11813, decmplen=20364, crc=9CA2F026
ver 2.0 myfile.zip/packageInfo.plist PKZIP Encr: cmplen=390, decmplen=433, crc=5B49A74F

Note: It is assumed that all files in each archive have the same password. 
If that is not the case, the hash may be uncrackable. To avoid this, use option -o to pick a file at a time.

To do that, I have used:

zip2john.exe -o SomeDir/SomeFile.itv myfile.zip > hash.txt

Then I tried the following command to search for the pass:

john.exe myfile.zip hash.txt

john told me:

Warning: Invalid UTF-8 seen reading myfile.zip
Warning: detected hash type "HMAC-SHA256", but the string is also recognized as "HMAC-SHA512"
Use the "--format=HMAC-SHA512" option to force loading these as that type istead
Warning: only loading hashes of type "HMAC-SHA256", but also saw type "PKZIP"
Use the "--format=PKZIP" option to force loading hashes of that type instead.

I could open the zip file in 7zip, so I assumed that I should go with "PKZIP".

I am now using the following command to search for the pass:

john.exe myfile.zip hash.txt --format=PKZIP 

I have opened the hash.txt file, and there is a huge string in it. Its length it over 23.000 characters.

It starts with

myfile.zip/SomeDir/SomeFile.itv:$pkzip2$1*1*2*0*2e25*4f8c*9ca2

and ends with

(...)3366b8e749b403b58dac12fcbf73359240*$/pkzip2$:SomeDir/SomeFile.itv:myfile.zip::myfile.zip

Is the length of over 23.000 characters normal?

I thought hashes were rather short.

Edit: I didn't expect that in this case, multiple hashes are involved. Had I know this, I would have posted the entire contents of hash.txt.

The is the full contents of "hash.txt":

myfile.zip:$pkzip2$21108249ca2aec6a18a34344111501fe327f342344ca0266159d1d7771112d212fa125f22edbc14201861b15b49a74f2e512f81865b49762df53ca761860e134342347b8471192cc4d2d8053a4a459aadf711b9de840e971d794b53434a39e7aed762a0515cd152e75cb14c33f4eb5c87d37dd65fc5e0df80398e00f9fd4405099f0ee0001967a47a2afca8ca8b30e3fbe7f60c981dd866bd8553b0ef5175c4ab7ae3e54280b005237a217e3c815cf02485b8aff14dcd5a3fb41b259317892491061f203ff39968b599ffa020b0184fd7d42d880624ce278c5933845fc01c9d64b42e34be3b4112087ca9473edafc997fe8be0602fbb6edc2e1650c276e3fbc70b59fd765fe6a6734e588332ee343440e101c05a22f9b6253919d743ad0a895cfc0f291534342105cdffb3e6cdc239a7b23a98bd8cf9dca8d18b67232434ea8eb4622af60018357d2e94670f93756c1183d6541310af9fa15c99ad4898acebd8081e4f8901cfaf417f43ddc5ded1434862dfc7f78a8039b128f5aac6c8794a5eb3e18660a75b0f23eb2c54c32117ff279def30a3a61e78af355f80f50d08c3b5026e493499c092a745fc6e34341a3cdf726d1bc*$/pkzip2$::myfile.zip:packageInfo.plist, SomeDir/SomeFile.itv:myfile.zip

Answer 1

You are correct regarding your general assumtion that hashes normally are not 23.000 characters and have a fixed length that does not depend on the input. In fact, most cryptographic hash functions are designed to take a string of any length as input and produce a fixed-length hash value.

pkzip2 "Hash" format

In the case of the used pkzip2 files there is a difference though. The pkzip2 files do not contain a hash of the password. It can therefore not be extracted as a hash and then cracked like one normally would. To guess the password, zip2john extracts data and checksums from the zip file and combines that in a predefined format. That format is subject to change but currently simiar to (see "Output Line Format" in the source https://github.com/openwall/john/blob/bleeding-jumbo/src/zip2john.c#L29):

The newer $pkzip2$ addressed the problem by adding TC (as in timestamp), but still neither zip2john or the format really knew when to use which (resulting in suboptimal early rejection). filename:$pkzip2$CB[DTMT{CLULCROFOX}CTDLCSTCDA]*$/pkzip2$

Command

The command to crack a pkzip2 file should be

john.exe hash.txt

Note that the original file must not be part of the command or else john will try to use "lines" in the file and try to detect hash formats. This results in john not beeing able to detect your hash format correctly. If you only supply the output of zip2john you will not have to specify the format manually.

Cracking the "hash"

The tool john then "tries a wordlist to decrypt the data stored at the DA position in zip2john's output, runs CRC32 on the result and compares that checksum with the checksum stored at the CR position".

Because the "hash" contains some of the original data, it can become quite long. The tool tries to use the smallest file in the archive to reduce the decryption work (and thereby also the length of the "hash").

This information was explained by magnum, the main contributor of the JohnTheRipper tool (https://www.openwall.com/lists/john-users/2021/06/03/2).

First of all, zip2john simply picks the smallest file from the archive in order to minimize the work. Sometimes there is no small file though.

Second trick: pkzip archives also include another, smaller, checksum (just 1 or 2 bytes) of a tiny part of the packed data of each file. If that doesn't match, we can early reject. But if it does match (and unfortunately a single byte checksum will randomly match in about 1 of 256 cases) we do have to checksum the whole (potentially large) file.

So third trick in our bag is we collect several such smaller checksums (for up to 8 files), and require all of them to match before going on and calculate a whole-file CRC. If the number of files is smaller, we may end up with less than 8 of course. On a side note, theoretically the files in one archive may use different passwords and that invalidates this trick (in fact we'll probably get false negatives). That is fairly uncommon so by default we assume all have the same password.

So best case for us is an archive with 8 or more files, and at least one small file. Worst case is an archive with only one, huge, file.