I have cryptographic code used in the Windows platform, which uses the Crypto API functions and need to convert this to using Common Crypto on OS X.
Essentially the original code is this, with error checking removed for brevity: -
CryptAcquireContext(&m_hProv, NULL, NULL, PROV_RSA_FULL, CRYPT_VERIFYCONTEXT)); CryptCreateHash(m_hProv, CALG_MD5 ,0, 0, &hHash); CryptHashData(hHash,(LPBYTE)pszInputData, lstrlen(pszInputData)*sizeof(TCHAR), 0); CryptDeriveKey(m_hProv, CALG_RC4, hHash, CRYPT_EXPORTABLE | 0x00280000, &m_hKey); CryptDecrypt(m_hKey, 0, bFinal, 0, pData, pdwDataSize);
As far as I understand, this is what's happening: -
CryptAcquireContext - Get an object to handle the cryptography
CryptCreateHash - Create an MD5 hashing object
CryptHashData - Hash the input data with MD5
CryptDeriveKey, CryptDecrypt - Decode pData with RC4, using the key m_hKey
The size of pszInputData is 12 bytes and the output array of the MD5 hashed object is the same on both platforms.
To decode with RC4, I'm doing the following with Common Crypto: -
CCCryptorRef cryptor = NULL; CCCryptorCreate(kCCDecrypt, kCCAlgorithmRC4, 0, (void*)m_hKey.data(), m_hKey.length(), NULL, &cryptor); char outBuffer; size_t outBytes; CCCryptorUpdate(cryptor, (void*)pData, *pdwDataSize, outBuffer, 12, &outBytes);
Testing the output (outBuffer array) from Common Crypto with an online RC4 decoder matches, so this is decoding correctly.
However, the final output from the Windows code in pData doesn't match the RC4 decoded in Common Crypto.
Are there some steps I'm missing or not understanding with the Windows Crypto API calls here; why do the outputs differ?
(Please note, I'm not looking for comments on the security or flaws in using RC4)