The attack is not stack overflow, but buffer overflow (the buffer being possibly located on the stack). The stack is a RAM area which applications use to store local variables and to organize the calling and returning from functions. A stack overflow is when the application code has gone way too recursive, and runs out of stack space. Since the kernel keeps unmapped pages at both ends of the stack space, stack overflow and stack underflow trigger traps, converted by the kernel to
SIGSEGV, implying ruthless termination of the offending application. This is a bug, but not often a security vulnerability.
A buffer overflow is when the application code is induced into writing into a buffer (a given area in RAM) more bytes than can possibly fit in said buffer, thus writing some of the bytes "elsewhere", on whatever was in RAM immediately after of immediately before the buffer. When the buffer is on the stack, what lies along the buffer is other local variables, and return addresses. A return address is written on a stack slot when a function is entered: it is the address where execution should continue when the function returns (i.e. it points to the instruction in the caller's code which immediately follows the
call opcode). By overwriting a return address (with a buffer overflow), the attacker will try to make code jump elsewhere.
In the classic times of buffer overflows, the attackers used the easy path: overflow of a buffer with attacker-chosen data; the data itself contains some code that the attacker would like to see executed (e.g. code which launches a shell); the overflow is used to overwrite the return address so that it points precisely to that data in the buffer; when the function returns, it follows that altered return address and jumps to the attacker's code.
A first protection measure is to mark the stack, and possibly most of the rest of the RAM, as non-executable. This is called Data Execution Prevention. It is about preventing the CPU from interpreting the data bytes (provided by the attacker) as code which could be executed. The hardware can make this distinction (on old 32-bit x86 CPU, it was not completely easy to do, but still possible).
In reaction, attackers began to use code which is already there in RAM, instead of inserting their own code chunks, e.g. by jumping into some function in the standard library (the standard library has many functions, some of which being quite convenient for the attacker). The standard library contains, by definition, code, so it is marked "executable" and DEP cannot do anything against that.
In reaction, defenders have devised Address Space Layout Randomization: this is what
va_randomize_space activates. This shuffles around dynamically-loaded libraries in RAM, so that the attacker cannot predict where in RAM the standard library is -- if he cannot predict the location of his target code, he will have a hard time jumping to it.
In reaction, attackers have began to target the main executable itself, or the dynamic linker. Both are at a fixed position in the address space. It so happens that most code chunks can be used by the attacker to do his nefarious schemes ("Return-Oriented Programming" has been coined for that). We are still waiting for the next step in that attacker-defender dance.
Canaries try to defeat the classical exploits of buffer overflows by preventing the use of an altered return address. That's what the
stack-protector option of GCC does (it does a few other things, but that's its main job). A special randomized value (the "canary") is written in the stack just before the return address slot; when the function returns, it first checks that the special value is still there. Presumably, if an overflow had reached the return address slow, it would also have written over the canary, modifying it; the code will see that, and abort, killing the application instead of jumping madly through the address space.
The canary makes any good only if the sole attacker's target is the return address of the current function; it also assumes that the overflow is not an underflow, and that it consists in a continuous sequence of bytes, and that the attacker "does not get lucky".
The core ideas in all these systems are that:
- They do not prevent overflows. They just try to make the consequences of such overflows less problematic. Canaries try to get some early notice of an overflow; ASLR is more about scrambling around to avoid the attacker's control jump.
- They concentrate on attack paths. DEP, ASLR, canaries... are not generic protection systems, but countermeasures to specific exploits seen in the wild. Instead of defeating attackers, they more or less train attackers into more creativity.
- They deal only with code path alteration, when the attacker tries to make the CPU jump to another code location. They do nothing about data alteration. With an overflow on the stack, you can also alter local variables. Depending on the application code, you may do other things, such as, for instance, modify a file descriptor so that the contents of a private file are not written to the said file but to another file or a socket or logs.
- Canaries are only about exploiting an overflow (not an underflow) of a buffer on the stack. But there are also buffers in other parts of RAM (e.g. the heap).
The real protection against buffer overflow is not to allow the data to overflow. ASLR, canaries... cope with some specific consequences, but that's like countering a flood by using a mop. To disallow buffer overflows, two ways:
- Be a very good programmer, and don't make bugs.