To prevent buffer overflows, there are several protections available such as using Canary values, ASLR, DEP, NX. But, where there is a will, there is a way. I am researching on the various methods an attacker could possibly bypass these protection schemes. It looks like there is no one place where clear information is provided. These are some of my thoughts.

Canary - An attacker could figure out the canary value and use that in his buffer injection to fool the stack guard from detecting an exploit

DEP, NX - If there are calls to VirtualAlloc(), VirtualProtect(), the attacker could try to redirect code to these functions and disable DEP, NX on the pages that he wants to inject arbitrary code on.

ASLR - No clue . How do ASLR and DEP work?

  • This 2014 paper describes a technique they call Blind Return Oriented Programming (BROP) that is like ROP but doesn't even require knowledge of the source or binary.
    – dramzy
    Jan 30, 2016 at 22:44

3 Answers 3


Stack canaries work by modifying every function's prologue and epilogue regions to place and check a value on the stack respectively. As such, if a stack buffer is overwritten during a memory copy operation, the error is noticed before execution returns from the copy function. When this happens, an exception is raised, which is passed back up the exception handler hierarchy until it finally hits the OS's default exception handler. If you can overwrite an existing exception handler structure in the stack, you can make it point to your own code. This is a Structured Exception Handling (SEH) exploit, and it allows you to completely skip the canary check.

DEP and NX essentially mark important structures in memory as non-executable, and force hardware-level exceptions if you try to execute those memory regions. This makes normal stack buffer overflows where you set eip to esp+offset and immediately run your shellcode impossible, because the stack is non-executable. Bypassing DEP and NX requires a cool trick called Return-Oriented Programming.

ROP essentially involves finding existing snippets of code from the program (called gadgets) and jumping to them, such that you produce a desired outcome. Since the code is part of legitimate executable memory, DEP and NX don't matter. These gadgets are chained together via the stack, which contains your exploit payload. Each entry in the stack corresponds to the address of the next ROP gadget. Each gadget is in the form of instr1; instr2; instr3; ... instrN; ret, so that the ret will jump to the next address on the stack after executing the instructions, thus chaining the gadgets together. Often additional values have to be placed on the stack in order to successfully complete a chain, due to instructions that would otherwise get in the way.

The trick is to chain these ROPs together in order to call a memory protection function such as VirtualProtect, which is then used to make the stack executable, so your shellcode can run, via an jmp esp or equivalent gadget. Tools like mona.py can be used to generate these ROP gadget chains, or find ROP gadgets in general.

There are a few ways to bypass ASLR:

  • Direct RET overwrite - Often processes with ASLR will still load non-ASLR modules, allowing you to just run your shellcode via a jmp esp.
  • Partial EIP overwrite - Only overwrite part of EIP, or use a reliable information disclosure in the stack to find what the real EIP should be, then use it to calculate your target. We still need a non-ASLR module for this though.
  • NOP spray - Create a big block of NOPs to increase chance of jump landing on legit memory. Difficult, but possible even when all modules are ASLR-enabled. Won't work if DEP is switched on though.
  • Bruteforce - If you can try an exploit with a vulnerability that doesn't make the program crash, you can bruteforce 256 different target addresses until it works.

Recommended reading:

  • 4
    Excellent post, minor question: I've never heard the term "NOP spray" but it seems equivalent to a "NOP sled" or "NoOP sled" which more closely describes the action of sliding over the intervening memory to get somewhere good. Is that one and the same as you're referring to, or is there a subtle distinction that makes "NOP spray" a separate thing? Thanks...
    – gowenfawr
    Sep 21, 2012 at 16:04
  • 1
    Yep, a NOP spray and a NOP sled are exactly the same thing. I've just been playing with heap sprays recently, so I went for the terminology that was on the top of my head.
    – Polynomial
    Sep 21, 2012 at 17:40
  • @Polynomial : In your answer here :security.stackexchange.com/questions/18556/… you had mentioned hardware enforcement of NX. So, will bypassing DEP using ROP (using VirtualProtect()) still work if NX bit is hardware enforced ?
    – sudhacker
    Sep 23, 2012 at 16:43
  • Yes. The NX bit just provides hardware enforcement of the executable memory page protection flag. With NX disabled, it's trivial to execute pages that don't have the executable bit set, because the processor won't recognise the difference. With NX enabled, you cannot execute non-executable pages. However, VirtualProtect alters the protection flags on the page, so it actually is marked as executable, which the CPU will then allow.
    – Polynomial
    Sep 23, 2012 at 21:22
  • @Polynomial : It seems it don't apply to heap overflows, isn’it ? Nov 1, 2015 at 23:48

Canaries and other volatiles do not prevent the overflow; they just try to cope with the consequences of an overflow which has happened. The canary tries to detect the case of an overflow which overwrote the return address in a stack frame. DEP is one step further, it assumes that the return address has been overwritten and followed, and it restricts the areas where execution could jump. ASLR is yet one step further: it "shuffles around" the areas where execution is allowed.

Historically, buffer overflows where exploited to overwrite the return address in the stack, so as to make execution jump into the very data which has been used to overflow the buffer. The canary tries to detect that before jumping, and DEP is used to make the stack space non-executable. DEP also works when overflowing buffers in the heap (the canary is of any use only for stack buffer overflows, but heap can contain buffers as well, and also sensitive data to overwrite, such as pointers to functions -- especially in the context of OOP languages such as C++). To work around DEP and the canary, attackers have begun to look for overflows which allow to overwrite pointers to function, so as to make execution jump into standard library code which is necessarily "there" and also necessarily executable. That's why ASLR was invented: to make such games harder. ASLR can still be defeated by being lucky: since ASLR must maintain page alignment (4 kB on x86), within a not-too-large address space (typically less than 2 GB on 32-bit x86), there are not so many places where the target code may be (at most half a million). Depending on the attack context and how often the attacker's script can try, this can be too low for comfort.

The important theme here is that canaries, DEP and ASLR do not defeat overflows themselves, but target the generic overflow exploit methods which have traditionally been employed. In any application, an overflow which overwrites non-pointer data can be as deadly as a remote shell exploit (e.g., imagine an overflow which modifies a string field called "authenticated_user_name"). The weapon race between attackers and defenders is becoming too specialized and, in my opinion, increasingly misses the point. On a general basis, it is much better to never allow the overflow to take place, i.e. block/kill the offending process/thread before writing bytes outside of the target buffer. That's what happens with almost any decent programming language (Java, C#, VB.NET, Python, Ruby, Node.js, OCaml, PHP... the choice is large).

  • 3
    +1 for solving the actual problem (programming languages which permit overflows to happen). You might want to mention that many scripting languages (looking at you, Python) have modules written in C for performance, especially modules which deal with the network (and therefore are potentially exposed to overflow)
    – devnul3
    Sep 27, 2012 at 18:31
  • @devnul3 : I’d be very glad to get a pure python implementation of zlib for example. (that’s because of that kind of things most security ideas of ibm as/400 are dying) Nov 1, 2015 at 23:44
  • 1
    @user2284570 I would prefer a pure Rust implementation. I personally think that C and C++ are obsolete for this reason alone. The C++ Core Guidelines might be able to fix this, IF they can create a useful subset of C++ that can be mechanically proven free of undefined behavior.
    – Demi
    Feb 13, 2017 at 21:04
  • @Demi : I was saying that because I was needing python. Feb 13, 2017 at 22:01
  • @user2284570 One could expose a C API to the Rust library.
    – Demi
    Feb 13, 2017 at 23:21

The basic level of protection is ASLR + DEP.

If you don't use both of those, then there are many powerful techniques for exploiting a buffer overrun (e.g., return-oriented computing, heap spraying, repeated guessing). For instance, DEP alone can be defeated using return-oriented computing; and ASLR alone can be defeated using heap spraying and repeated attempts.

However, if the target uses both ASLR + DEP, exploitation becomes significantly harder. The techniques mentioned above are not sufficient to defeat ASLR + DEP. ASLR + DEP are like a one-two punch that make the attacker's life much harder. Defeating the combination of ASLR + DEP is not impossible, but it takes much more cleverness.

My favorite example of methods for defeating ASLR + DEP is explained in the slide deck, Interpreter Exploitation: Pointer Inference and JIT Spraying. There, the author describes how he exploited a memory safety error in Flash. He exploited properties of the Flash JIT to arrange memory in a way that lets him mount a code-injection attack, despite the presence of ASLR + DEP. Recall that a JIT is a just-in-time compiler; it compiles Flash bytecodes to native code. The native code will be stored somewhere in memory, and the Flash JIT marks it executable (despite DEP). The author found a way to generate Flash bytecodes that, when compiled, would generate a sequence of bytes that embedded his malicious shellcode (offset by a byte). He then used heap spraying techniques to ensure that there were many copies of this in memory. Finally, he exploited the memory-safety bug to cause the program to jump to another address; due to ASLR, this was like jumping to a random address, but the many copies ensured that with high probability this would jump into his shellcode. In this way, he bypassed both ASLR and DEP -- a nifty feat.

One last note: it is worth mentioning that ASLR is much more effective on 64-bit architectures. On 32-bit architectures, ASLR can often be defeated by simply making multiple attempts. There just aren't enough degrees of freedom on 32-bit platforms to introduce enough randomness, so the attacker's chances of succeeding by dumb luck remain too high, on 32-bit platforms. For the strongest defense, use a 64-bit platform.

  • My case : Canaries, ASLR, NX combined. long mode with System 5 abi. The vulnerability happen with an overflowing array which is allocated on heap. Is remote code execution possible ? Nov 2, 2015 at 1:20
  • @user2284570, please don't use comments to ask a new question. This should perhaps be asked as a separate question (but make sure to search and research before asking, and show your research in the question -- there's lots written on how to bypass ASLR, so make sure you read the standard resources before asking and frame your question accordingly). Likely the answer will depend upon the specifics of the particular application you are attacking.
    – D.W.
    Nov 2, 2015 at 5:55
  • Ok, so in general (for staying on topic) does this prevent remote code execution based on heap overflow ? Nov 2, 2015 at 11:46
  • @user2284570, everything in my answer applies to heap overflows. As my answer says, ASLR+DEP does not always absolutely prevent remote code execution but makes it harder or sometimes impossible.
    – D.W.
    Nov 2, 2015 at 16:44

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .