What common techniques are there to decrypt a file given that I only know the private RSA key?

Question

I've been trying to solve an online challenge. I've been given a file, which I suspect is encrypted (around 15KiB). I've also managed to obtain what seems to be an RSA private key (n == 2048). I know that RSA wasn't used to encrypt the whole file since from what I've read, given that n = 2048, the maximum data size it can encrypt is 245 bytes.

What are my options here? I've read that usually large files are encrypted using a symmetric encryption algorithm and that RSA is used to encrypt the symmetric key.

Given that I only have the encrypted file and the private RSA key, what is my best course of action to try and decrypt the file? Are there any common techniques here?

Answer 1

Take the first 256-bytes and try decrypting them using the key? Or the last 256? You'll know if it doesn't work because the padding will be incorrect (try with both PKCS#1 v1.5 padding and OAEP). If it works, you can then see whether what you get out is something like an AES or 3DES key which could be used to decrypt the rest of the data.

You could also try running the file through an ASN1 decoder to see whether it is in any other format that is encoded that way.