I'm writing a client app in Java that sends encrypted data back to the server. All clients have my RSA public key. Is there a standard way to encrypt a given file with it to send it back to the server?

From websites such as this one, I gather that usually, a symmetric key is generated to encrypt the actual file, and this key is then encrypted with the RSA public key and sent along the file. Is there a standard way in Java or a standard file format (without additional libs) to package the encrypted symmetric key and the symmetrically encrypted file?

Of course, I could design my own file format, but I'd rather stick to standards if possible.

  • 1
    BouncyCastle maybe? Its the only library I know of that does encryption
    – TheLQ
    Commented Jan 4, 2011 at 15:11
  • 1
    How would I do it with BouncyCastle? Commented Jan 4, 2011 at 15:51
  • 3
    What is your underlying goal and threat model? E.g. do you also want to authenticate where the data came from, or are you vulnerable to a MITM attack? Have you considered SSL and then encrypting the file on the server side if desired?
    – nealmcb
    Commented Jan 4, 2011 at 15:53
  • 1
    +1 to @nealmcb, it shouldn't be assumed that file encryption is necessarily the way to go.
    – AviD
    Commented Jan 4, 2011 at 18:58
  • The client app saves user activity logs whose content should be published to a server unaltered, even if the user has read/write access to the saved files. Moreover, the user should not be able to inspect the contents (the files are exercise, progress, and error reports from a quiz application). Commented Jan 4, 2011 at 19:51

4 Answers 4


One common, standard encryption format is Cryptographic Message Syntax (CMS, evolved from PKCS#7), and e.g. BouncyCastle supports it. It allows for signatures also, and is the basis, e.g., for S/MIME mail message security. It may be overkill, but is widely supported.

Example code is provided by Bouncy Castle. For CMS, see org.bouncycastle.cms.test, probably the Enveloped Data examples.

Another option is the XML Encryption format, standardized by W3C. For Java examples, see e.g. Exploring XML Encryption, Part 1 or do it with XSS4J: Implementing XML Encryption in Java


The simplest and most surefire way of file encryption is to shell out and invoke gpg or pgp to encrypt it. I know it sounds inelegant, but in my experience reviewing a lot of crypto code, this is the simplest way to ensure that your solution is secure, without having to know a lot about cryptography. If you want to do it yourself, you will need to learn more about how cryptography works, and the risk of a security problem increases.

Make sure there is authenticity protection for the encrypted data (e.g., encrypt and then sign/MAC): you will need both confidentiality and authenticity protection. A very common error is to encrypt but fail to sign; this error can introduce subtle security problems. (If you are not a cryptographer, I realize this may not make any sense to you. I realize this is by no means obvious, but I am a cryptographer. If you are familiar with IND-CCA2 and INT-CTXT, you will understand why I am recommending this. If you're not familiar with those concepts, you can read about them if you want, or you can take my word on it. Hey, crypto is tricky stuff; what can I say.) For instance, the website you link to makes this mistake; I should not rely upon that website for advice about secure encryption. The strawman approach you mention in your question also suffers from this flaw, and it looks like some of the proposed answers here also have a similar shortcoming.

An alternative approach, which is a very good one if the client and server are connected in real time, is to open a TLS-encrypted channel between the client and the server and send the data over that channel. Make sure that the client verifies the public key / cert of the server. If you want to authenticate clients, give clients a client cert and make sure that the server requires the client to provide a client cert and validates that cert.

  • 4
    1. Adds requirement that the gpg/pgp tool is installed on the client box. 2. You now have a new problem: you need to ensure the integrity of a tool you're calling via exec().
    – user185
    Commented Jan 7, 2011 at 9:47
  • 1
    @Graham, yes, this requires gpg/pgp. No, this is not a new problem: you already have to ensure the integrity of your code, so use the exact same mechanism for gpg/pgp. In practice integrity is not hard. Note: I didn't say my solution has no drawbacks, I said it was the simplest way to get good security. If you have a simpler way to do it securely, feel free to post it. But be warned in advance that there are many tricky details.
    – D.W.
    Commented Jan 8, 2011 at 0:39
  • 1
    @D.W. but you have a race condition. if(pgpExecutable.isValid()) { exec(pgpExecutable.path(), ...)
    – user185
    Commented Jan 8, 2011 at 10:31
  • 1
    @Graham, you are simply wrong. That is not the solution I was proposing, nor does my solution have a race condition or any other such problem. Sounds like you're a bit confused about the threat model and how software is deployed to end machines.
    – D.W.
    Commented Jan 9, 2011 at 1:05
  • 1
    @Graham, again, you are wrong. As I already stated, you are attacking a strawman. I am not proposing what you think I am proposing.
    – D.W.
    Commented Jan 12, 2011 at 2:41

@JPP, This comment of yours makes me think you're asking the wrong question:

The client app saves user activity logs whose content should be published to a server unaltered, even if the user has read/write access to the saved files. Moreover, the user should not be able to inspect the contents (the files are exercise, progress, and error reports from a quiz application).

If the client app writes the log files, and encrypts them - that means the client app has access to the encryption key, whatever that may be.
However, you're trying to prevent the user of the client app from reading the files!
That's just not going to work, because:

  1. The user has access to anything the client app has access to - including the file contents, and the encryption key...
  2. The user has control of anything running in his own process - i.e. he can even attach a debugger to the client app - so don't bother trying to get smart and figure out a way to obfuscate the encryption key.

To me, your problem sound more like one of access control - how can I write a log file from a client app, without the user having access to the contents?
One way would be to build a local service (e.g. Windows Service, on Windows) that runs in a different user context; the client app would talk to the service, and the service would then write the contents to the file in a protected location that regular users cannot reach.
Another alternative is, as @D.W. suggested, send it directly to the server immediately, and not write to file any of the contents.

The problem with both these solutions, is as I previously noted: The user has control of anything in his process, and can block or modify any requests sent, either to the server or the local Windows service.

  • I don't see the problem if I use my public certificate to encrypt what the client writes to disk, and then use my private key to decrypt it once I have received it. Commented Jan 12, 2011 at 11:25
  • 1
    @JPP, even assuming that the private key is not available on the client machine, but rather only on the server, note also point #2 above, wrt e.g. attaching a debugger - if the malicious user is determined to subvert the process, he will be able to. Thats not to say that encrypting with the standalone public key wont add value - it does raise the bar somewhat, but just so long as you realize thats ALL it does, and it can be gotten around.
    – AviD
    Commented Jan 12, 2011 at 11:30
  • Thanks for the explanation. Sure, attaching a debugger and directly modifying the byte array before it's encrypted is possible. But what's the alternative? Discard all user data generated this way if he's not able to communicate with the server right now? And even then: I can also attach the debugger and modify the byte array before the client sends it to the server, or before it gets sent to a secure Windows Service, so the situation's not better. Or am I missing something? Commented Jan 12, 2011 at 11:39
  • 1
    No, @JPP, not missing something - except for the very basic fact that you cannot rely on the client. Overall, and given the constraints, the encryption as you laid it is probably the best, but take it for what its worth. It will prevent tampering with the file, but during processing its all open. If that is a problem, you'd be better of redesigning your system so the sensitive processing is done on the server.
    – AviD
    Commented Jan 12, 2011 at 12:00

Apparently, there's no way to do it with the standard library without significant recoding of stuff already available in Bouncy Castle, for instance.

Here's what I came up with. On the client side, I encrypt a file plainfile as follows:

// install security provider
if (Security.getProvider(BouncyCastleProvider.PROVIDER_NAME) == null) {
    Security.addProvider(new BouncyCastleProvider());

// load public certificate for signing
KeyStore publicKs = KeyStore.getInstance(KeyStore.getDefaultType());
publicKs.load(new FileInputStream("/path/to/publicKeystore.ks"), null);
TrustedCertificateEntry entry = (TrustedCertificateEntry) publicKs.getEntry("public", null);
X509Certificate cert = (X509Certificate) entry.getTrustedCertificate();

// create CMS envelope data;
// check http://www.ietf.org/rfc/rfc3852.txt pages 15-16 for details
CMSEnvelopedDataGenerator envelopedDataGen = new CMSEnvelopedDataGenerator();

// specify that generated symmetric key will be encrypted by with this public key

// automatically generate the AES key, encrypt the data, and encrypt the key
CMSEnvelopedData data = envelopedDataGen.generate(new CMSProcessableFile(plainfile),
        CMSEnvelopedDataGenerator.AES256_CBC, 256, BouncyCastleProvider.PROVIDER_NAME);

byte[] encryptedData = data.getEncoded();

Then, on the server, I can decrypt this data as follows:

// load private key
KeyStore privateKs = KeyStore.getInstance(KeyStore.getDefaultType());
privateKs.load(new FileInputStream("/path/to/privateKeystore.ks"), null);
PrivateKeyEntry privateKeyEntry = (PrivateKeyEntry) privateKs.getEntry("privatekey",
        new KeyStore.PasswordProtection("password".toCharArray()));
PrivateKey privateKey = privateKeyEntry.getPrivateKey();

byte[] encryptedData = ...

// parse CMS envelope data
CMSEnvelopedDataParser envelopedDataParser = new CMSEnvelopedDataParser(new ByteArrayInputStream(encryptedData));

// expect exactly one recipient
Collection<?> recipients = envelopedDataParser.getRecipientInfos().getRecipients();
if (recipients.size() != 1)
    throw new IllegalArgumentException();

// retrieve recipient and decode it
RecipientInformation recipient = (RecipientInformation) recipients.iterator().next();
byte[] decryptedData = recipient.getContent(privateKey, BouncyCastleProvider.PROVIDER_NAME);

Please leave comments if I'm missing something important or am making mistakes here... Thanks!

  • A file with such encrypted contents ciperfile.enc can be decrypted with openssl: openssl cms -decrypt -in cipherfile.enc -inform DER -recip testcert.pem -inkey testkey.pem, where testcert.pem is the certificate in the public keystore in PEM format, and testkey.pem the corresponding private key. Commented Jan 5, 2011 at 9:30
  • 2
    I'm concerned that this code snippet does not seem to provide message authentication/integrity protection. If so, it is likely to be insecure against active attacks, and thus not a recommended solution.
    – D.W.
    Commented Jan 8, 2011 at 0:37
  • 1
    @D.W. Thanks for pointing this out. How can I fix it? Sign it after encryption? Commented Jan 12, 2011 at 11:30
  • 1
    Yes, signing after encrypting would provide authentication/integrity, if done right. However this is tricky. The best way to fix it is to rely upon some existing, well-vetted solution for file encryption, like PGP/GPG, or for channel encryption, like SSL/TLS.
    – D.W.
    Commented Jan 19, 2011 at 18:28
  • 1
    See this post for further explanation of why it is a bad idea to encrypt without also authenticating and how to fix the problem. It is worth emphasizing that the sample code listed above appears to be flawed and should not be used.
    – D.W.
    Commented Feb 28, 2011 at 9:19

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .