Hardware DEP takes advantage of the NX ("No Execute page protection",
AMD specification) or XD ("Execute Disable", Intel specification) bit
on DEP compatible CPU’s, and will mark certain parts of the memory
(which should only contain data, such as the default heap, stack,
memory pools) as non-executable.
When an attempt is made to execute code from a DEP protected data
page, an access violation (STATUS_ACCESS_VIOLATION (0xc0000005)) will
occur. In most cases, this will result in process termination
(unhandled exception). As a result of this, when a developer decided
he wants to allow code to run from a certain memory page, he will have
to allocate the memory and mark it as executable.
http://www.corelan.be/index.php/2010/06/16/exploit-writing-tutorial-part-10-chaining-dep-with-rop-the-rubikstm-cube/#hwdep
When you're trying to redirect code flow through a buffer overflow exploit, etc you want to redirect the flow to a custom shellcode/payload that will give you remote access, etc to the victim computer.
We do this by re-writing registers like EIP(Extended Instruction Pointer), ESP(Extended Stack Pointer), etc. By this technology we attempt to prevent redirecting code flow to unauthorized memory locations. Hence in order to invoke the shell, we'd need to get past the access violation in theory.
Essentially programmers have to specify memory locations that can have executable code, the other locations have non-executable data on invoking which causes an access violation exception.
Read more about its technicalities here: http://msdn.microsoft.com/en-us/library/bb736299(VS.85).aspx
