Let’s say I have the following pseudocode in the trusted part of a sandbox which prevent untrusted code calling mprotect() and mmap() and ptrace() directly (mutext isn’t accessible from sandboxed memory)

//src and dest are user controlled but must be valid.
TrustedPart_for_safe_jit(void * mutext, uint8_t *src,uint8_t *dest, uint32_t size) // in the current case, *dest targets a PROT_NONE memory region
    ValidateOpcodesOrCrash(src,size); // uses calls to mmap on size internally. Contains many different loops and use several 10k thousands lines of codes in the trusted part of the sandbox : this is the longest part. Please also note that src is write protected while being in this function.
    unwriteprotect(dest,size); // calls many sandbox’s internal functions

    SafeMemcpy(src,dest,size); // THIS IS the function which contains the race condition

    unEXECprotect(dest,size); // involve write protecting as well as allowing reading

SafeMemcpy(uint8_t *src,uint8_t *dest, uint32_t size) // the data to be copied cannot exceed 128Mb
    if(!CheckUserTargetPointToValidMemroyRange(dest,size) {
        uint8_t *src_ptr=src;
        uint8_t *dest_ptr=dest;
        uint8_t *end_ptr=des+size;
        while (dest_ptr < end_ptr) { // that loop should execute very fast
            *(uint32_t *) dest_ptr = *(uint32_t *) src_ptr;
            dest_ptr += 4;
            src_ptr += 4;

That part is responsible for allowing untrusted code to use ᴊɪᴛ compilation.
The point is untrusted thread aren’t suspended.

As you know, when 2 threads use memcpy() with the same destination, they generate random data. In that case, such data could potentially contains instructions like int 0x80, thus allowing to escape the sandbox.

Things I thought to so far :

  • Write data to a file and read it with the read system call through the sandbox. If the memory is still write protected the program doesn’t crash. This would involve looping and even if the data to be copied can be 128Mb large I’m not sure it would works because of syscall overhead.
    An Alternative would be to create code several times and try reading with several timing, but I have no idea on how to select the initial timing window.
  • Use futex… But I couldn’t found if futex can be used to check the value of non allocated memory. I’m also unsure if the thread could wake up before memory become write protected.

So, is it possible to plan the timing window for memcpy race conditions ?

  • 2
    Are you trying to write to the memory from a different thread before it gets write protected?
    – grc
    Feb 20, 2017 at 6:09
  • @grc : Yes, I want to write after and before it is write protected. The aim is to start memory writing during thewhile (dest_ptr < end_ptr)loop. As modern ᴄᴘᴜ typically copies data inside such loops at 19Gb/s, the timing window is very short. Feb 20, 2017 at 12:49
  • If you're copying 128Mb, I think the timing window would be relatively large compared with syscalls.
    – grc
    Feb 20, 2017 at 13:56
  • @grc : yes, but I can create 128Mb at max (the maximum size allowed for the dynamic code region). If I want to create dynamic code several time, I have to use smaller size (I recognize It’s strange but they documented that dynamic codes region can’t be deleted securely because of race condition). Also what timing to choose ? I can’t access to the real fileystem, so I have no way to know about ᴄᴘᴜ speed. Feb 20, 2017 at 15:04
  • How do you get the interefering thread running on this system? I assume it must previously be copied using the given mechanism, and then allowed to execute in the sandbox. Before being copied it must pass ValidateOpcodes(). Does ValidateOpcodes() remove or reject data for copy? If so what does it not allow?
    – this.josh
    Mar 1, 2017 at 7:34

2 Answers 2


Based on what I see here, you can modify *src after ValidateOpcodesOrCrash finished checking that part of the memory but before SafeMemcpy starts.

I don't know how ValidateOpcodesOrCrash is implemented, but presuming that it simply loops through [src,src+size] and look for illegal instructions, then you can call with a fairly TrustedPart_for_safe_jit large size, busy wait for a few hundreds of CPU cycles, and then start overwriting *src that ValidateOpcodesOrCrash likely had finished checking. If ValidateOpcodesOrCrash does something more complicated, you can figure out what sequence of instructions will be the fastest and slowest for ValidateOpcodesOrCrash to check, put the fastest at the front and, a whole lot of the slowest instructions all up to the end. You probably won't need to wait for ValidateOpcodesOrCrash to complete before you start overwriting src.

  • wait for a few hundreds of CPU cyclesHow do you define that when the hardware could be the the best lastest 64 bits Xeon with ddr4 or an old amd opteron with ddr ? I also can’t see why I should not modify src while inSafeMemcpy. The only thing done byCheckUserTargetis to check the address of dest is in allowed memory range. Mar 21, 2017 at 13:08
  • what sequence of instructions will be the fastest and slowest for ValidateOpcodesOrCrashThe problem on x86 is opcodes that are more complicate to decode also takes more space. Simple ones likenopsare decoded very fast, but you can use up to 15nopwhere a single and more complicate opcode would take the same space. Also I can’t modifysrcbeforeValidateOpcodesOrCrashfinish due to write protection being applied while processing. Mar 21, 2017 at 13:24
  • @user2284570: You can tweak the amount of busy loop manually, all we'll be trying to do here is to increase the attack window to increase your odds. An exploit don't need to be 100% reliable. A defender can only fail once, an attacker have nearly infinite retries. If the team developing this software don't understand this concept, they shouldn't be writing any security software.
    – Lie Ryan
    Mar 21, 2017 at 14:17
  • @user2284570: another thing you might want to try is to do lots of memory access and writes in your other threads you control, to increase memory bandwidth contention with the supervisor thread. The goal here is to try to force the supervisor thread to exhaust its time slice in the middle of SafeMemcpy and get preempted/context switch, which should give you a fairly large time slice to exploit the race condition, and hopefully be able to implant your exploit code.
    – Lie Ryan
    Mar 21, 2017 at 14:24
  • Thanks, but this still doesn’t tell if 10,000,000 cpu cycles would be largely too much or too few depending on the context (remember if the timing window is incorrect the program crash). Because of crash if invalid I can’t know if the timing was too short or too long. Also, the sandbox requires executables to be downloaded from an http server (and there’s no caching). They can only be triggered by the user from local source if he/she had fully accepted to install the to be sandboxed program. This makes retrying costly. I feel that it’s like guessing an address in the case of aslr. Mar 21, 2017 at 14:41

(Disclaimer: I am not a security professional – just a student with some knowledge of C)

I think that this is likely exploitable, at least to cause a crash. To fix the bug, you need to either:

  • suspend all untrusted threads while dest is writeable.
  • or have TrustedPart_for_safe_jit return freshly allocated executable memory (I assume the untrusted code is not allowed to cough up arbitrary machine addresses as pointers).

That means that instead of using a mutex, you need to stop-the-world.

  • The whole design of the sandbox is to crash the process in the case of trying to execute non allowed code. This is a sandbox : the aim is to allow executing any code programmer want like with javascript. In fact if I want to perform crashing I can just access non allocated memory. I have full access to pointers, provided they are shrunk to 32 bits values (so accessing important data require 64 bits pointers, the lower 32 bits only contains validated code and data). Mar 19, 2017 at 19:18
  • suspend all untrusted threads while dest is writeable.The point of my question is I have no ideas on how to determine when it is writable… If I could, I could just start memcpy writing. Mar 19, 2017 at 19:19
  • @user2284570 The problem is that MutexLock() needs to suspend all write access to src or dst from any thread. That is why you need to suspend all untrusted threads.
    – Demi
    Mar 19, 2017 at 19:23
  • The problem is that MutexLock() needs to suspend all write access to src or dst It doesn’t. It just preventmutextto be locked. I mean that mutext is global data except that it can’t be accessed from untrusted code. The only thingMutexLock()prevent is to get 2 threads callingTrustedPart_for_safe_jitat the same time, which would allow to have the correct timing. The point isdstis required to be read_only beforeTrustedPart_for_safe_jit. srcremains always readable and writable, but how to determine the correct timing for modifyingsrcafterValidateOpcodesOrCrash()finished ? Mar 19, 2017 at 19:48
  • or have TrustedPart_for_safe_jit return freshly allocated executable memory Though I didn’t wrote it, you could have guessed it : that since it’s in memory accessible from untrusted code to allocate read only data (TrustedPart_for_safe_jit exit if the data is writable since the begging). The only way to create executable memory is to useTrustedPart_for_safe_jit. Once data as been allocated, it’s status can’t be changed except if by freeing memory. The exception is jit code which can never freed without restarting the process (that’s why TrustedPart_for_safe_jithas no real use). Mar 19, 2017 at 19:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.