To a large extent, the question makes no real sense, because the data is opaque for the server. Since the server will not, in any case, be able to read the decrypted files, whether two files are identical or not should not have any impact. In fact the complete scenario must be made clearer.
You have a server. The server stores "encrypted data" on behalf of some users who will send to each other the decryption keys; all decryption and encryption occurs elsewhere, and your server cannot do it. We have to assume that there is an out-of-band mechanism by which users exchange decryption keys.
What you want, as the server's owner, is to recognize when several instances of the same file are uploaded to your server, so that you may pay for the storage space only once (I just read somewhere that "The Hobbit" was the most copied movie of 2013, so you might assume that your envisioned platform would contain several hundreds of copies of that multi-gigabyte movie file). Normal, secure encryption should prevent outsiders from being able to make any assertion about the data contents, including being able to recognize whether two encrypted files have the same contents. In that sense, the de-duplication you are looking for is a weakening of the security model. We can predict that some users would not like it; for instance, if de-duplication works, then you (as the server) can detect when a user is uploading "The Hobbit".
(When de-duplication works, then it becomes possible to do some exhaustive search on file contents; this is easy to do for files which are duplicated a lot, because a file which is known to many users cannot be really that secret; and in the context of "intellectual property enforcement", heavily-duped files are of prime interest.)
If we speak in generic terms, then you cannot enforce de-duplication, because any two users who want to evade it can simply double-encrypt their data. User 1 first encrypts the file with an extra secret key, that he will share with User 2; that key is chosen randomly, so the encrypted file won't match any other instance of the same data; that encrypted file is then encrypted again with your service. Users can do such double encryption because the data is just, in fine, a file on their disk, and we assumed above that users can talk to each other independently of your server. As long as users can talk to each other, then they can exchange secret keys that you don't know of.
In some very specific contexts, it is possible to prove that some encrypted data fulfills some algebraic property without revealing it. See non-interactive Zero-Knowledge proofs. This is used in some electronic voting protocols, so that the prover may demonstrate that what he encrypted is really either a 0 or a 1, not any other integer, but without divulging the actual vote value. This won't apply to a generic file storage system, because "normal files" don't follow a discernable mathematical structure.
Therefore, if you want to enforce de-duplication against uncooperative users, then you must make it so that users cannot talk to each other. That's easier said that done... and it looks hardly feasible, since for the site to be really useful, the sender and recipient must be able to agree on at least a reference key, used to locate the file on the server.