I am developing an application which will run untrusted code. I have been designing it for linux, thinking that I can just run it in a virtual machine on windows when the need arises. The linux security features are that the untrusted code is in an interpreted language running under seccomp, with syscalls mediated by a seccomp-like utrace policy, inside a restrictive linux container, and under a restrictive AppArmor profile.

A thought just occurred to me about running this virtualized under windows, though: suppose a hostile program manages to break the interpreted language and execute arbitrary machine code within the virtual machine. Can it make direct system calls to the host windows OS, bypassing the linux access policies? If so, I will need corresponding sandboxing policies in windows itself, something I am quite ignorant of. Are there windows features corresponding to seccomp, utrace, linux containers, and AppArmor?

  • 3
    Are you asking if code running in a linux guest can break out of a VM to infect the windows host? See How secure are virtual machines? and Are there ways to protect the guest kernels at the hypervisor level?
    – bstpierre
    Commented Dec 16, 2011 at 13:05
  • Thanks for the feedback. No, I am aware that hostile code can break out of a VM. My question is whether OS access controls in a sandbox running on the guest OS are secure against hostile code in the sandbox gaining access to the host OS through machine code syscalls which the guest OS does not recognize as such because the guest and host syscall conventions are different. Please let me know if there's any way I can edit the question to clarify this, or if there's a better forum for this question. The threads you linked do not address my question. Commented Dec 16, 2011 at 13:36
  • I think your comment clarifies the question. I only linked those questions because they sounded similar to your original question. There's probably someone here who can give you a good answer.
    – bstpierre
    Commented Dec 16, 2011 at 13:39

2 Answers 2


The linux security features are that the untrusted code is in an interpreted language running under seccomp, with syscalls mediated by a seccomp-like utrace policy, inside a restrictive linux container, and under a restrictive AppArmor profile.

... crikey. In a good way.

Can it make direct system calls to the host windows OS, bypassing the linux access policies?

Well, the answer to this depends on how you are visualising and a few other factors - which really needs a "how does virtualisation work" explanation to cover it.

Assuming Intel-VT, I've covered in a bit of depth (plus links) virtualisation here. As a very brief summary, your host does the usual operating system things - page tables, call gate decriptors. Then to run a virtual machine the host sets up another table for describing vms including a memory map for the vm and then runs a VMXON instruction - and then the VM is created.

From the perspective of the virtual machine, the memory it is presented is the whole system memory available to it - the same idea as virtual memory for a normal application. Under normal virtualisation scenarios you do not present the same memory your host runs on to the guest - you give it its own space which means it cannot directly reach any of your existing setup.

However, if you could map host memory to the guest, to make a host system call directly you'd need to be able to raise an interrupt registered with the host. Here's a fairly good internals of NT document - it's different enough from Windows that to actually implement this would probably require modifying your kernel and I'd be reasonably confident in saying that if you exposed the memory as is, you'd either break Windows, or you'd break Linux. This is, of course, assuming you map your host memory into the guest virtual machine... I might not be being very clear, so to say exactly what the problem is - linux has registered int 80 (or is using sysenter); Windows is registering int 2e or sysenter (another good article - same author as the previous one)). They could definitely register different software interrupt handlers if either had the other free... but sharing sysenter would be impossible without modification - and that's just a problem with memory locations. Things get even stickier when we talk about calling conventions...

We're not quite out of the woods yet though. Whilst you cannot directly easily without some serious engineering to modify the platforms involved call the host's system calls clearly there needs to be some way to communicate with the host some of the time. There is - a VMCall instruction (in the guest) causes a VMExit to the host with info exchange so the host can provision a request - if there are any vulnerabilities here, you might indirectly be able to call a system call, or call(s). These happen whenever you and the host need to share something other than the CPU and memory. Intel VT-d is an effort to provide DMA as needed in hardware, circumventing the need for VMCalls for some classes of device.

Things are slightly different if you do software virtualisation. To provide the same thing you have to write a CPU emulator - however, this is a host-running process and thus if you can inject code and execute it, you can call a system call in the context of whatever permissions the emulator has. This is comparable (very loosely, sort of, disclaimer: metaphorically speaking) to a bug in the JVM - a malformed piece of java bytecode that can inject shellcode into the VM is going to do nasty things, no matter the security provided by the JVM.

I should add the obvious here - I'm talking about engineering and virtual machine problems. If you set up a samba share and double click dodgy.exe that was provided by your malicious code, it's game over. It sounds like that's an obvious statement for you - but just in case other people think VMs solve all their problems... not quite.

Various references:


It would be very difficult (to the point of verging on rebuilding parts of the OSes) in hardware-based virtualisation to directly call a system call belonging to the host from the guest; even more so when they use different OSes. Exploits are possible indirectly, but then require vulnerabilities in the hypervisor code or its usage.

  • 1
    That was incredibly informative, Ninefingers. Thanks. Is there windows virtualization software (preferably open-source) which allows fine-grained control over how it responds to a VMCall? The ideal would be something which would allow nothing more than simple communication with my trusted process, like seccomp is supposed to. Commented Dec 16, 2011 at 19:26
  • I don't know - I expect not, but then some of the VM solutions (VirtualBox) are open source. The thing with the VMCall is the guest has to be aware that it is being virtualised and can use the VMCall - think "Guest Additions" type stuff you installed in your VM. The other type of hook is your "VMExit" conditions for things you need to support - and these are everything you might need to run - emulation for network hardware etc. It'd be a hard work to evaluate/restrict/lock that down because you'd probably kill a lot of functionality you need.
    – user2213
    Commented Dec 16, 2011 at 21:56
  • That said - if you were prepared to host on Linux - libvirtd/qemu-kvm on Fedora contains SELinux labelling for virtual machines, keeping each VM isolated via category labels - in theory. Again, though, if you're looking for assurance you'd still need to verify that this was actually the case... honestly, using seccomp+apparmor inside a VM has probably put enough defence in place! The other alternative if you really fancy it is to take your VM platform of choice and start evaluating the source, removing everything you don't need.
    – user2213
    Commented Dec 16, 2011 at 22:00
  • My questions might seem paranoid, but if the application becomes popular will be there will be a fair bit of money in compromising it, so I want to think hard about the security of it from the start. I'm thinking about Windows at the moment because I want to find a way to be comfortable about it running on random stranger's machines. Commented Dec 17, 2011 at 0:20

The short answer is "No" (barring vulnerabilities/bugs in the hypervisor code). Guest code cannot invoke system calls in host OS, except by breaking out of the VM. That's just how virtual machines work. Guest code running inside the virtual machine does not have the ability to directly invoke host OS system calls, except by exploiting some vulnerability in the hypervisor to break out of the VM. Modern hypervisors are designed to prevent this from happening, and while bugs/vulnerabilities in the hypervisor are not impossible and have happened, they are rare.

I'm actually a bit puzzled by the question, so I hope I've answered the question you had in mind. Once we remove the distractors and inessential details, I understood you to be asking "Can malicious guest code invoke the host OS and ask it to perform a system call?" If that is what you meant to ask, then the answer is "No", as virtual machines just don't provide any way for the guest code to invoke host OS system calls. If that's not what you wanted to know about, please elaborate.

  • 1
    I think the OP's question was in the context of that break out
    – Rory Alsop
    Commented Dec 19, 2011 at 20:45
  • @Rory, That's not how I read it. Can I persuade you to read the original post again? The original poster posited a system that is executing untrusted code (in some interpreted language) using an interpreter, which is itself inside a VM. The OP asked: what if there is a vulnerability in the interpreter which allows malicious code to break out of the interpreter? At that point the malicious code has full ability to execute arbitrary guest code in the guest VM, but not to break out of the VM. In short: the OP was in the context of a break-out of the interpreter, not a break-out of the VM.
    – D.W.
    Commented Dec 20, 2011 at 9:04
  • I was reading the "manages to break the interpreted language and execute arbitrary machine code within the virtual machine" piece as key
    – Rory Alsop
    Commented Dec 20, 2011 at 9:07
  • @Rory, I think you're mis-reading the question. The interpreter is a separate mechanism from the VM. "Executing arbitrary machine code within the virtual machine" means arbitrary guest code, still inside the VM. Breaking the interpreted language does not mean you somehow magically escape the VM, as they're two separate mechanisms. (Example: say I run a Python program in a Linux guest, running in VMWare on a Windows host. If the Python program is malicious and attacks the Python interpreter, it's still confined to within the Linux guest and can't harm the Windows host, barring VMware bugs.)
    – D.W.
    Commented Dec 20, 2011 at 9:13
  • 1
    No - I agree with what you are saying regarding the interpreter, but the OP's comment "execute arbitrary machine code within the virtual machine" shows we are looking at ways to escape a VM, which @Ninefingers' answer does describe.
    – Rory Alsop
    Commented Dec 20, 2011 at 9:29

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .