I need to encrypt large dataset files that may be in gigbytes of size. I can perform compression task on the dataset too. Which will be a good algorithm to use that is fast enough as well as secure enough?

  • 1
    "fast enough ... secure enough" - there is no common understanding of enough in this context, i.e. it depends on your specific requirements regarding speed and security. Even if the meaning of enough is clear the answer would also depend on the available hardware: some algorithms perform well with hardware acceleration while others are better if software-only implementation is required. Therefore I propose to close the question as too broad. Commented May 28, 2017 at 14:45

1 Answer 1


Depends on what you're trying to secure it from. I would say that, generally speaking, AES-128 is a good compromise between fast and strong, and there's a lot of hardware support for AES available, which helps to speed it up.

I assume you're not trying to protect it from a state actor or someone with similar resources who want to gain access to your data? If so, you might want to consider longer key sizes and/or chaining other encryption algorithms.

  • I am trying to secure data from being decrypted without key(and hence permission). Commented May 28, 2017 at 14:35
  • Sorry, I meant who you had in mind for the type of person you are trying to prevent from gaining access to the data. For example, if you are trying to protect medical records, you might want to use stronger encryption than if you are trying to protect a web site password database, on the basis that whoever wants access to the medical records is likely to put more resources towards decrypting it, and that the consequences of losing medical data is generally much worse than the consequences of losing a password database.
    – Pak
    Commented May 28, 2017 at 15:09
  • As hinted in my previous comment, you'll also need to consider what value the data has for anyone who has unauthorised access to it, and what the consequences of that might be. Basically, it's a decision made knowing what the value of the data is, what threats there are, and what the consequences of losing it are.
    – Pak
    Commented May 28, 2017 at 15:17
  • The data is sensitive as it contains a few rows of people's identifiable information. Are there any standards for big data security in storage & transit? Commented May 28, 2017 at 15:42
  • I basically have to distribute dataset to some users. I want to check what can be done to make distribution of dataset as secure as possible so that it doesnt get stolen while being transferred or after transfer is complete. Commented May 28, 2017 at 15:44

Not the answer you're looking for? Browse other questions tagged .