5

I was wondering how do hackers use cryptography for bypassing antivirus software.

I don't have some deep understanding on how antivirus software works, but I know that every antivirus has base of signatures, so when we scan .exe file it actually compares signature of .exe with signatures in it's own base, and if it matches, antivirus will identify .exe as virus.

So by applying some cryptographic algorithm on .exe file (using tool "Crypter"), they change signature of .exe, so antivirus can't detect it as malware, right? But concretely, how do they apply cryptograpyic algorithm on .exe file? I saw only in my book how plaintext is crypted.

Improve this question
3
  • 1
    For a computer system everything is a binary blob. If you have seen cryptography applied to a text file and all its contents have changed (encrypted), apply the same crypto to an executable binary blob and surely everything will change. AV signatures won't work. The only difference is you will attack the decryption stub to the start of the exe (the initial entry point) so that the encrypted payload is encrypted at runtime for execution.
    – void_in
    Commented Jun 27, 2015 at 15:31
  • Hi void_in, thanks for reply. I studied few cryptographic algorithms like Rijndael, DES,... and I perfectly understand how they work (for any text as input I know how to get ciphertext using these algorithms), but how these algorithms are applied to input which is .exe? And that is not only one problem to me. Let's say I encrypted .exe (on some way) I want to hide from antivirus. I will get some "unreadable and undetectable file", if I'm not wrong. So to run that file there must be another file to decrypt it, but If I decrypt it I will come to original file (which is detected by antivirus)?
    – etf
    Commented Jun 27, 2015 at 15:45
  • 1
    There is no difference between an exe and an ASCII file as far as encryption algorithms are concerned. If you have studied AES, you will know the input is 16 bytes of data. This data could be anything: ASCII text, binary blobs from an exe file, anything. So encrypting an exe is the same as encryption any other type of data. For decryption, the decryption routine is executed first in memory and that is why AVs fail to detect it. The signature engine check the file BEFORE execution and not during execution.
    – void_in
    Commented Jun 27, 2015 at 16:12

2 Answers 2

Reset to default
2

Executable file encryption is rather obfuscation than real encryption, since encrypted file still have to execute. So "decryptor" part has to know the decryption algorithm and/or key.

Antivirus software has:

  • various emulators, for x86 code, x64 code, normalized JavaScript code etc.

  • various decryptors/unpackers for all "crypters" and archives known to antivirus vendor (eg. ESET has 1228 decryptors/unpackers as for now)

As for your question: basically signature matching is done after unpacking, decrypting etc., when antivirus engine finds out, that there are no more known archive/encryption/obfuscation levels for current file.

However there are 2 exceptions:

  1. There are modular viruses, eg. Flame, in which only one module has decryption key for another module, which has decryption key for another module and so on. Catching such viruses require writing dedicated code in antivirus engine.

  2. There are ransomware viruses, that, while not being encrypted themselves, can encrypt random files on infected computer in a way that nobody but virus author has decryption key. Files encrypted by such viruses are not recoverable by antivirus software.

Improve this answer
4
  • Thanks for reply. Besides that one has to know some programming language for this, what other skills should one have? Is it neccessary to have deep understanding of how Windows works (especially how .exe work, ie. what is happening when application is executed etc)?
    – etf
    Commented Jun 30, 2015 at 16:21
  • 1
    To write a good virus, you have to know, how computer works on hardware level, how compiler works (with low level details), how Windows kernel works, have very good understanding of various Windows APIs and antivirus engines. Fortunately there is lot of materials: Windows NT4 and 2000 source code leaked about 12 years ago, Kaspersky Antivirus and Norton Antivirus source code leaked a few years ago, and of course open source ClamAV and Windows SDKs. On the other side, this is too much for one person, that's why the best viruses (from already catched) are believed to be written by whole teams.
    – Tomasz Klim
    Commented Jun 30, 2015 at 17:09
  • Do you have some suggestion about literature covering these fileds? I found "Windows Internals 6th Edition by Mark Russinovich, David Solomon and Alex Ionesc"
    – etf
    Commented Jul 1, 2015 at 0:00
  • 1
    I can recommend anything written by Mark Russinovich, however it's definitely not enough. Search also for books about hardware architecture (focused on how CPU really works), and for something about LLVM - this will give you some knowledge about recent compilers. Then, prepare to dig into source code...
    – Tomasz Klim
    Commented Jul 1, 2015 at 0:39
1

To put it simply, an attacker can create an executable that has an encrypted virus inside of it. When the executable is downloaded to the victims machine it is not detected as a virus because the bad part is encrypted. When the user clicks on the file, The outer executable will load the encrypted data in memory and decrypt it, then execute it, all using RAM so it never touches the hard drive. The antivirus won't detect it because of this unless it has run time protection. Depending on the antivirus used it is often possible to bypass runtime detection my calling thread.sleep('10'); or something like that. When the executable is clicked on, it waits 10 seconds and then executes the encrypted file.

Improve this answer
1
  • Thanks for reply. Besides that one has to know some programming language for this, what other skills should one have? Is it neccessary to have deep understanding of how Windows works (especially how .exe work, ie. what is happening when application is executed etc)?
    – etf
    Commented Jun 30, 2015 at 16:25

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .