Bufferoverflow example on 64 bit Linux

Question

I am attempting to learn about buffer overflows. I have written a simple C program that calls a function called checkpw() which allocates a char array of 10 bytes large. When I analyze the op code I see that it actually allocates 16 bytes on the stack.

My goal was to overflow that buffer and write over the return pointer address, which I have successfully done. I have pointed it to a bit further down the buffer where I have shell code, where the first instruction is xor eax,eax.

During the GDB disassembly I can si single step through the op codes successfully until the rip (instruction pointer) points exactly where I want it to point (0x00007fffffffde40).

I even see the => when I examine that memory location with the x/i 0x00007fffffffde40 which then displays the op code xor eax,eax.

The problem is as soon as I engage in the next instruction with si I get a SEGFAULT.

When I compiled the program, I made sure to include the switch -fno-stack-protector setting, and I also performed echo 0 > /proc/sys/kernel/randomize_va_space

Other than that, I am stuck. Why I am getting a SEGFAULT instead of executing the xor?

Answer 1

You're running into memory protection, specifically the NX bit. On modern systems, memory is typically flagged as being either data (and thus can be read and written) or as code (and thus can be read and executed). Attempting to write to a code area or execute a data area will trigger a SEGFAULT.