I've tried to replicate the workflow presented on this blog in OpenSSL: http://farid.hajji.name/blog/2009/07/27/public-key-cryptography-with-openssl/
However, authentication seems to fail despite many variations. What's going wrong? Please see the code below you can copy and paste into OpenSSL. Note that I am using the compiled Windows binary version of OpenSSL.
//================Phase 1 - Setup================ //Generate my private key (myprivatekey.txt) genpkey -algorithm RSA -out C:\myprivatekey.txt -pass pass:abc123 -pkeyopt rsa_keygen_bits:2048 //Generate friend's private key (friendprivatekey.txt) genpkey -algorithm RSA -out C:\friendprivatekey.txt -pass pass:123abc -pkeyopt rsa_keygen_bits:2048 ---------------- //Extract my public key (mypublickey.txt) from my private key (myprivatekey.txt) rsa -passin pass:abc123 -in C:\myprivatekey.txt -pubout -out C:\mypublickey.txt //Extract friend's public key (friendpublickey.txt) from my private key (friendprivatekey.txt) rsa -passin pass:123abc -in C:\friendprivatekey.txt -pubout -out C:\friendpublickey.txt ---------------- //Generate my password (a random base64 string password saved mypassword.txt) rand -base64 -out C:\mypassword.txt 128 //Generate friend's password (a random base64 string password saved to friendpassword.txt) rand -base64 -out C:\friendpassword.txt 128 //Delete the .rnd file that's generated? Not sure what it is. ---------------- //Encrypt my password using my private key (encrypted password saved to a binary file - myencryptedpassword.txt) pkeyutl -in C:\mypassword.txt -out C:\myencryptedpassword.txt -inkey C:\myprivatekey.txt -passin pass:abc123 //Encrypt friend's password using friend's private key (encrypted password saved to a binary file - friendencryptedpassword.txt) pkeyutl -in C:\friendpassword.txt -out C:\friendencryptedpassword.txt -inkey C:\friendprivatekey.txt -passin pass:123abc ---------------- //Convert my encrypted password to base64 from binary (saved as myencryptedpasswordbase64.txt) base64 -in C:\myencryptedpassword.txt -out C:\myencryptedpasswordbase64.txt //Convert friend's encrypted password to base64 from binary (saved as friendencryptedpasswordbase64.txt) base64 -in C:\friendencryptedpassword.txt -out C:\friendencryptedpasswordbase64.txt ---------------- //Create a signed hash of my password so my friend knows it's coming from me (signed hash saved as mysignedhash.txt and is in binary form) dgst -sha256 -sign C:\myprivatekey.txt -passin pass:abc123 -out C:\mysignedhash.txt C:\myencryptedpasswordbase64.txt //Create a signed hash of friend's password so I know it's coming from my friend (signed hash saved as friendsignedhash.txt and is in binary form) dgst -sha256 -sign C:\friendprivatekey.txt -passin pass:123abc -out C:\friendsignedhash.txt C:\friendencryptedpasswordbase64.txt ---------------- //Convert my signed hash from binary to base64 base64 -in C:\mysignedhash.txt -out C:\mysignedhashbase64.txt //Convert friend's signed hash from binary to base64 base64 -in C:\friendsignedhash.txt -out C:\friendsignedhashbase64.txt //================Phase 2 - Authentication================ //Now, we reverse the process and authenticate the friend. Let's prefix all output files with "phase2" //I provide friend with my public key and my encrypted password //Friend provides me with their public key //Convert friend's encrypted password from base64 to binary. The output file will be the same as friendsignedhash.txt base64 -d -in C:\friendsignedhashbase64.txt -out C:\phase2friendsignedhash.txt //Convert friend's signed hash from base64 to binary. The output file will be the same as C:\friendsignedhash.txt base64 -d -in C:\friendencryptedpasswordbase64.txt -out C:\phase2friendencryptedpassword.txt //Verify if the password originates from my friend (by checking against my friend's public key) dgst -sha256 -verify C:\friendpublickey.txt -signature C:\phase2friendsignedhash.txt -out C:\friendresult.txt C:\phase2friendencryptedpassword.txt
Any idea of why the verification failure occurs?