My understanding of virtual memory is that the operating system allows each process to have access to the entire pool of memory, and creates this illusion by paging.

As a consequence, a program can only see it's own memory, and can not see the real system memory, and certainly not the memory space of other applications.

How do exploits such as buffer overflows work (when there is no input to the program), if they can't see outside of their own memory?

I'm talking about the type of exploits that you compile and that result in an executable that exploits a daemon (and not by passing specially crafted input through human input fields).

  • 1
    Don't forget that virtual memory was not originally a security feature. Also, what you're describing sounds more like sandboxing than standard virtual memory. – Matthew Mar 5 '16 at 13:41
  • @Matthew, I'm not meaning to talk about sandboxing or any technology that deliberately hides memory for security. It is just that as per my understanding, when a program executes it can't see the memory of other programs. Is this wrong? – Tim Matheson Mar 5 '16 at 19:15
  • I'm not 100% certain, but I suspect it's not entirely correct. For example, if you run task manager in Windows, it can see the memory use of other programs - this isn't quite the same as seeing the memory contents, but is a well known program. Procdump (technet.microsoft.com/en-gb/sysinternals/dd996900.aspx) can dump the memory contents of a process - by definition, it must be able to see the contents. – Matthew Mar 5 '16 at 19:29
  • Exploits such as a buffer overflow only need to affect the program's own memory in order to exploit the program. – Macil Apr 6 '16 at 19:06

I'm talking about the type of exploits that you compile and that result in an executable that exploits a daemon (and not by passing specially crafted input through human input fields).

They do so by passing specially crafted input through non-human input fields. A program that does something useful needs to interact with some other programs in some fashion, be it through network protocols, through interprocess communication protocols, by reading and writing files, etc. The malicious program uses whatever interaction method the demon uses and sends it specially crafted input. As is the case with input from humans, the demon parses the input. Due to the bug in the demon that makes the exploit possible (the vulnerability), this input is not processed correctly, but instead leads to executing the payload (the “useful” code contained in the exploit).

If you think about it, human input fields go through intermediate programs. When you exploit a web application, for example, the exploit is delivered through the remote operating system's network layer, to the web server, to the application. Exploits are hardly ever delivered by a human typing at a keyboard, and even if they are, the exploit is delivered through the operating system's keyboard driver, to the GUI subsystem if there is one, and so on.

| improve this answer | |

A buffer overflow is a type of vulnerability not an exploit. It will overflow into the virtual memory of the process that is executing it.

"One process cannot access another process' memory". This is entirely Operating System dependent. In Windows/Linux/Android this is a true statement. There are lots of access controls in place to prevent a process from tampering with memory it shouldn't need access to. However, lots of embedded systems will allow any process to write to any portion of memory.

A daemon is a service that can be triggered by an external probe. If there is a vulnerability in a daemon then the malicious binary can simply spoof a probe that will trigger the vulnerability. Hard-coded buffers can be used to trigger it while hard-coded shell code is used to exploit the vulnerability. If the daemon is exploited then the shell code is run in order to gain execution. The daemon is no longer in control of the code it is running; the malicious input buffer is now in control. And since the daemon processed the input data, the malicious input buffer is running within the context of the daemon process.

Why would a malicious binary exploit a service? Generally it would be to gain higher level privileges. The malicious binary might be able to run at low level privileges, and then through the service gain administrative privileges. This can be used to ensure that it persists across reboots or attempts to evade Personal Security Programs.

| improve this answer | |
  • 1
    I appreciate the answer, but I know what the terms I used meant. Buffer overflow exploits are referred to as....buffer overflow exploits, so that's shouldn't have been ambiguous given the context, although I should have been more clear. That aside., what I ask is OS dependent, and I had Linux/Windows in mind when I asked. You say it is a true statement that on these OSs a process cannot access another memory. What I am asking then, is how do some exploits that you compile, attack a running service or process that it should not be able to see? – Tim Matheson Mar 7 '16 at 20:23
  • For example, how does a code exploit that compile to an executable windows EXE file manage to exploit another process which is not listening on any ports or interfaces, and there should be no way to communicate with it. Such as exploits that attack the Java JRE? – Tim Matheson Mar 7 '16 at 20:26
  • In the case of the JRE the Java program itself is the input, and the JRE runs the code. If there is a vulnerability in how a JRE library, or how JRE interprets a Java program, then it can be exploited. Another example would be an implementation of a compression algorithm. If the compression implementation itself is flawed, then sending a malicious compressed file could take advantage of the vulnerability. – RoraΖ Mar 7 '16 at 20:45
  • So, in all cases an exploit must be able to communicate with the process in some way, and can never directly read the memory of the process it wants to write to? – Tim Matheson Mar 7 '16 at 22:23
  • Well, "never" is a strong word. There are always exceptions, kernel level exploits, tricking the OS into giving it memory (flaw in the overall design), but in general terms yes. Some type of processing must be performed by the target process on data that is controlled by the attacker. – RoraΖ Mar 8 '16 at 12:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.