There are plenty of answers available which detail how antivirus and anti-malware programs work, including details on signatures, heuristics, sandboxing, etc. However, there seem to be relatively few resources on how anti-exploit programs (such as EMET, MBAE and HitmanPro.Alert) actually detect and prevent exploits.

The actions that actually occur during an exploit seem to be almost impossible to detect without attaching a debugger to the process, which would appear to have unacceptable performance impacts.

What are the different methods used by anti-exploit programs to detect and mitigate exploit attempts?

  • Other options include compiler features and runtimes like java and .net.
    – Aria
    Commented Sep 24, 2016 at 12:37
  • I don't follow you... Aren't anti-exploit programs supposed to protect all applications in general anyway?
    – konsolas
    Commented Sep 24, 2016 at 14:50

2 Answers 2


I'll answer this based on my experience with the CBMC compiler. I never used HitmanPro, malwarebytes anti-exploit or Microsoft's EMET but I do not see any reason why they would do things completely differently from CBMC.

Known exploit protection

If we look at the malwarebytes anti-exploit video (kindly researched by OP), we see that MBAE could find an exploit by identifying a downloaded malicious file. That is exactly how a modern anti-virus program works: it checks the signature of a known malware (or "exploit payload" if you want to call it that way) and stops its execution if it matches.

In that specific case we can argue that Anti-Exploit is simply a re-branded name for an Anti-Virus program. In theory you can argue that AV (Anti-Virus) only checks signature of files, whilst AE (Anti-Exploit) checks the payload (e.g. whiled downloading a file) You can also argue that AEs searches for CVEs whilst AVs search for malware signatures. But all modern AVs perform all that: they are updated when CVEs come out, they check signatures of network traffic (downloaded files) and they do that in real-time. So I'll argue that, from that video, we can conclude that AEs are simply AVs rebranded.

Yet, there is more: AEs promise to be able to find zero-day exploits. And that would be different from AVs.

Zero-day exploits

You can't find a zero-day exploit in a program without running the program. But running a dodgy program is unwise. This is because, if there is an exploitable point in the runtime environment, running a program may exploit exactly that breach. Sandboxing is an option, but sandboxes often have breaches of their won. So, how can we execute a program without executing it? Symbolic execution to the rescue.

As I said at the beginning of the answer I'll use CBMC, a tool I needed at some point in the past. It performs symbolic execution, but it is a research tool. And no one really argues that symbolic execution can find all possible problems a program may have.

CBMC is a C compiler (and has a commercial version for VHDL too) that does not compile a program into machine code. Instead it compiles the program into a language called GOTO, which is a simplified symbolic representation of the program. Then a GOTO processor (part of CBMC) is used to run the symbolic representation of the program.

A symbolic representation is much easier to reason about, and the program can be run assuming a wide range of inputs. For example, one of the things that CBMC/GOTO catches well in C programs are buffer overflows.

Now, CBMC works on C and VHDL programs, but the same can be done with assembly. Although it would not be as accurate as constructing a symbolic representation from the source code (since disassembly is not perfect), it would allow to execute the symbolic representation of the program and check for coding mistakes.

Buffer overflows are the easiest things that one can think that can be found in the assembly as well, but other possibilities may include unchecked input (input is tracked in a symbolic representation, at least in CBMC) or opening network sockets (since it is a known syscall).

Disclaimer: I do not know how an AE is actually built, but if I had to build something that can find a zero-day exploit automatically symbolic execution would be the path I'd try. Symbolic execution does not come without issues (known to the open research community, some non-disclosed sources may have found solutions to some of them):

  • SE does not find all problems, e.g. interaction between different configurations is often not found.
  • SE is rather slow.
  • Intel or ARM (or any other) disassebmly is never perfect, and disassembly problems reflect into SE results.
  • Although CBMC-GC proved to be an interesting read, I don't think it answers the question. You suggest that an anti-exploit program would statically analyze a program before execution and patch areas allowing exploitation. I feel that this is both unrealistic (packers? anti-disassembly? network resources? external libraries?), and discordant with the behaviour of anti-exploit programs which typically terminate the application when they 'detect' an exploit attempt. This obviously wouldn't happen if it were able to successfully patch the exploit before execution.
    – konsolas
    Commented Sep 25, 2016 at 11:24
  • A demonstration of what MBAE does
    – konsolas
    Commented Sep 25, 2016 at 11:27
  • Some antivirus programs certainly also protect against certain exploit codes with their signature database (e.g. scanning javascript). Anti-exploit programs often flout their claim of being able to protect against zero-day attacks without any signatures: HitmanPro.Alert page. I'm interested in how such programs can detect things like heap spray, ROP chains, etc being executed without any of the performance hit that would come with attaching a debugger, or analyzing the program at startup.
    – konsolas
    Commented Sep 25, 2016 at 19:57
  • Again, CBMC seems to be a very interesting project, but it doesn't appear to answer the question about existing widespread anti-exploit programs.
    – konsolas
    Commented Sep 26, 2016 at 17:41
  • 1
    @konsolas - Heh, I'm as curious as you are :). I used the video you posted in the answer (I hope you do not mind) and made a huge edit which bumped the question to the front (that's kind of cheating but sec.SE is kind of dead during the weekend). Still I added the comparison of the video with a modern Anti-Virus, and I'm arguing that that presentation of Anti-Exploit programs is simply rebranding.
    – grochmal
    Commented Sep 26, 2016 at 18:26

There are different approaches that anti-exploit programs use. If you know how an exploit works and what kind of exploitation technique it uses, (ROP, Heap Spraying etc.) it's pretty simple to understand anti-exploit products.

Let's take "heap spraying" as an example. Here is a definition from Wikipedia; "In general, code that sprays the heap attempts to put a certain sequence of bytes at a predetermined location in the memory of a target process by having it allocate (large) blocks on the process's heap and fill the bytes in these blocks with the right values."

If you look at heap spraying exploits , mostly they spray the memory to higher addresses and then they change vulnerable program's execution to a predictable address like "0x0c0c0c0c". Yes this kind of address is actually that predetermined location in the memory that we allocate/spray with the help of Javascript in browsers.

So how an anti-exploit program can prevent this ? Here is a very simple and commonly used approach; Let's assume you would like to protect Firefox against heap spraying attacks. Whenever firefox process starts , you can inject yourself to it and then you can pre-allocate these kind of common heap spray addresses (0x0c0c0c0c) in Firefox. So if an heap spraying exploit that uses common predictable addresses (0x0c0c0c0c, 0a0a0a0a etc.) will not work against this protected firefox process. A simple API call to pre-allocate an address; VirtualAlloc(SprayAddress, 0x400, MEM_RESERVE,PAGE_NOACCESS)

This is just a basic example of how anti-exploit programs work. Of course there are much more complex approaches and techniques used to prevent other exploitation methods.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .