Linux kernel crypto algorithms: potential exploitation?

Question

I am interested in many of the crypto algorithms in the arch/x86/crypto branch of linux kernel:

http://lxr.free-electrons.com/source/arch/x86/crypto/

and this:

http://lxr.free-electrons.com/source/crypto/

Many of these algorithms like registers like AVX and XMM registers, which I am not sure if the kernel ALWAYs reinitialize it from previous context (very performance-heavy if implemented). (Please provide the reference to the kernel source). And so if not, if one kernel module is doing the calculation, and then the intermediate keys can be sniffed from a usermode application looking at the XMM registers, is it not a risk?

Moreover, if someone can modify these registers, it may corrupt the AES computation as well, if the computation can be interrupted and scheduling passed proessing to a usermode application.

To generalise the above two scenario further:

If process A were to use XMM register - be it in the usermode or kernel mode, during process switching to process B, these XMM registers values should be backup and switched. So if there were 10 processes all using XMM registers, then the context for each of them will contain the XMM information.
And since these have to be done in the kernel during switching, can someone help to point out the kernel source that show this is done?

A similar question to this is found here (unanswered):

https://stackoverflow.com/questions/10879546/user-to-kernel-transition-and-xmm-register-state

Answer 1

Many of these algorithms like registers like AVX and XMM registers, which I am not sure if the kernel ALWAYs reinitialize it from previous context (very performance-heavy if implemented)

Oh, but it does. At least, for processes that use SSE/XMM. Access to SSE/XMM registers can be disabled in hardware (via CR4.OSFXSR), which also makes task switching more efficient for processes that aren't using SSE/XMM at all.

And so if not, if one kernel module is doing the calculation, and then the intermediate keys can be sniffed from a usermode application looking at the XMM registers, is it not a risk?

The kernel is very careful to sanitize everything that could've been touched by the kernel code before returning to userspace.

It may not necessarily save/restore, in some cases it might just clear with a zero value. But everything gets either restored or sanitized.

Where this is done is a different question. The default assumption might be that the kernel doesn't touch such advanced registers, so the responsibility might lie with the modules that use them (at least for system calls; task switching can be slightly more involved than system calls).

Answer 2

OK, let's go through a stroll through kernel code. I agree to some extent with @DepressedDaniel that the kernel is likely very careful in cleaning up the registers, process switch is a critical moment in kernel code (possibly the most critical moment) therefore a lot of attention to make that code secure is made.

On the other hand, the MMX/SSE/SSE2/3DNOW/MMXEXT registers are floating point registers that are not often used. I'm placing all those extensions in the same boat since they're all simply long registers together with a handful of opcodes to operate on them. During process switch saving and restoring a bunch of floating point registers that are not even used is a big performance hit, therefore the kernel does not bother doing so unless it needs. The question then becomes "How the kernel knows that it needs to deal with the MMX registers?". That is something that should be present in thread.flags, we will get to that shortly.

After searching for the registers, I found that they're used in more places than I thought at first:

• The RAID code uses MM0-MM7, and you can even see where it uses MM0-MM15 on x86_64 machines (there's a non SSE implementation too).
• AMD's 3DNOW code is still in the kernel, although I doubt that is used on x86_64
• KVM uses the MMX sutff, since it may need to emulate it
• Funny enough, I could not find any MMX use in the encryption modules (I didn't search too hard though).

The process switch part, at the first look, does not seem to use the MMX registers. But that, of course, cannot be true since we well know that we can process switch the RAID or the KVM process out of the CPU and back. The main function of process switch is (and pretty much always has been) switch_to(). In there we can see:

#define switch_to(prev,next,last) do { \
     if (ia64_psr(task_pt_regs(prev))->mfh && ia64_is_local_fpu_owner(prev)) { \
             ia64_psr(task_pt_regs(prev))->mfh = 0;                  \
             (prev)->thread.flags |= IA64_THREAD_FPH_VALID; \
             __ia64_save_fpu((prev)->thread.fph); \
     } \
     __switch_to(prev, next, last); \
     /* "next" in old context is "current" in new context */ \
     if (unlikely((current->thread.flags & IA64_THREAD_MIGRATION) && \
                  (task_cpu(current) != \
                  task_thread_info(current)->last_cpu))) { \
             platform_migrate(current); \
             task_thread_info(current)->last_cpu = task_cpu(current); \
     } \
} while (0)

And the pt_regs struct does not contain anything about MM0-5, but contains stuff about MM6-11 (at the very end).

So, what gives? Where the hell the kernel deals with other registers? The key it the __switch_to() procedure (well, macro, if you want to be pedantic) that is a little higher in the file above:

#define __switch_to(prev,next,last) do { \
    if (IA64_HAS_EXTRA_STATE(prev)) \
            ia64_save_extra(prev); \
    if (IA64_HAS_EXTRA_STATE(next)) \
            ia64_load_extra(next); \
    ia64_psr(task_pt_regs(next))->dfh = !ia64_is_local_fpu_owner(next); \
    (last) = ia64_switch_to((next)); \
} while (0)

And inside ia64_save_extra() the thread flags are checked and dealt with.

Also note that in the above ia64_fph_disable() is always called. The code for the other ia64_fph_* procedures is just below the disable macro.

The final trick appears to be the fact that on x86_64 the MM0-5 registers are considered volatile by the x86_64 CPU, therefore they are saved together with RAX, RCX and RDX. On plain x86 this is probably done inside the code dealing with the MMX stuff directly (and sorry for the wikipedia link, I could not find the ABI docs).

Answer 3

Check out this branch: arch/x86/kernel/fpu (which handle all the x86's FPU specific stuff):

Read this comment that can provide the answer:

http://lxr.free-electrons.com/source/arch/x86/kernel/fpu/init.c

/*
 * FPU context switching strategies:
 *
 * Against popular belief, we don't do lazy FPU saves, due to the
 * task migration complications it brings on SMP - we only do
 * lazy FPU restores.
 *
 * 'lazy' is the traditional strategy, which is based on setting
 * CR0::TS to 1 during context-switch (instead of doing a full
 * restore of the FPU state), which causes the first FPU instruction
 * after the context switch (whenever it is executed) to fault - at
 * which point we lazily restore the FPU state into FPU registers.
 *
 * Tasks are of course under no obligation to execute FPU instructions,
 * so it can easily happen that another context-switch occurs without
 * a single FPU instruction being executed. If we eventually switch
 * back to the original task (that still owns the FPU) then we have
 * not only saved the restores along the way, but we also have the
 * FPU ready to be used for the original task.
 *
 * 'lazy' is deprecated because it's almost never a performance win
 * and it's much more complicated than 'eager'.
 *
 * 'eager' switching is by default on all CPUs, there we switch the FPU
 * state during every context switch, regardless of whether the task
 * has used FPU instructions in that time slice or not. This is done
 * because modern FPU context saving instructions are able to optimize
 * state saving and restoration in hardware: they can detect both
 * unused and untouched FPU state and optimize accordingly.

So to repeat what is explained:

a. LAZY mode: FPU is not restored/saved all the time, but only when it is used, and the use of FPU will also reset a flag in CR0:TS, thus we don't have to detect for FPU register usage all the time. But this mode is not the default, as the time save/performance enhanced is not significant, and the algorithm become very complex, thus increasing processing overheads.

b. EAGER mode: This is the default mode. FPU is always saved and restored for each context switch. But again there is hardware feature that can detect whether the long chain of FPU registers are used - and whichever are used, only that register will be saved/restored, and thus it is very hardware efficient.

To do this is no mean feat, as it meant writing 208 patches in 2015:

https://lwn.net/Articles/643235/

The instructions to save all FPU - XMM, MMX, SSE, SSE2 etc is called FXSAVE, FNSAVE, FSAVE:

http://x86.renejeschke.de/html/file_module_x86_id_128.html

and the overhead in linux kernel is benchmarked as 87 cycles.

https://lwn.net/Articles/643235/

These optimized way of saving can also be found in comments below:

 * When executing XSAVEOPT (or other optimized XSAVE instructions), if
 * a processor implementation detects that an FPU state component is still
 * (or is again) in its initialized state, it may clear the corresponding
 * bit in the header.xfeatures field, and can skip the writeout of registers
 * to the corresponding memory layout.
 *
 * This means that when the bit is zero, the state component might still contain
 * some previous - non-initialized register state.

To detect that the kernel are triggered on FPU usage, we can set breakpoint on fpstate_sanitize_xstate in KGDB, and the kernel stacktrace are as follows:

Thread 441 hit Breakpoint 1, fpstate_sanitize_xstate (fpu=0xffff8801e7a2ea80) at /build/linux-FvcHlK/linux-4.4.0/arch/x86/kernel/fpu/xstate.c:111
111 {
#0  fpstate_sanitize_xstate (fpu=0xffff8801e7a2ea80) at /build/linux-FvcHlK/linux-4.4.0/arch/x86/kernel/fpu/xstate.c:111
#1  0xffffffff8103b183 in copy_fpstate_to_sigframe (buf=0xffff8801e7a2ea80, buf_fx=0x7f73ad4fe3c0, size=<optimized out>) at /build/linux-FvcHlK/linux-4.4.0/arch/x86/kernel/fpu/signal.c:178
#2  0xffffffff8102e207 in get_sigframe (frame_size=440, fpstate=0xffff880034dcbe10, regs=<optimized out>, ka=<optimized out>) at /build/linux-FvcHlK/linux-4.4.0/arch/x86/kernel/signal.c:247
#3  0xffffffff8102e703 in __setup_rt_frame (regs=<optimized out>, set=<optimized out>, ksig=<optimized out>, sig=<optimized out>) at /build/linux-FvcHlK/linux-4.4.0/arch/x86/kernel/signal.c:413
#4  setup_rt_frame (regs=<optimized out>, ksig=<optimized out>) at /build/linux-FvcHlK/linux-4.4.0/arch/x86/kernel/signal.c:627
#5  handle_signal (regs=<optimized out>, ksig=<optimized out>) at /build/linux-FvcHlK/linux-4.4.0/arch/x86/kernel/signal.c:671
#6  do_signal (regs=0xffff880034dcbf58) at /build/linux-FvcHlK/linux-4.4.0/arch/x86/kernel/signal.c:714
#7  0xffffffff8100320c in exit_to_usermode_loop (regs=0xffff880034dcbf58, cached_flags=4) at /build/linux-FvcHlK/linux-4.4.0/arch/x86/entry/common.c:248
#8  0xffffffff81003c6e in prepare_exit_to_usermode (regs=<optimized out>) at /build/linux-FvcHlK/linux-4.4.0/arch/x86/entry/common.c:283

Using "info thread 441" (see above) and you will find out that "Xorg" is the originator of the above stacktrace, but otherwise, majority of processes does not use FPU.

From the stacktrace, "get_sigframe()" is the first function that seemed to analyze on FPU usage:

if (fpu->fpstate_active) {
        unsigned long fx_aligned, math_size;

        sp = fpu__alloc_mathframe(sp, 1, &fx_aligned, &math_size);
        *fpstate = (struct _fpstate_32 __user *) sp;
        if (copy_fpstate_to_sigframe(*fpstate, (void __user *)fx_aligned,
                            math_size) < 0)
                return (void __user *) -1L;
}

So essentially what is happening here is copying the FPU information to userspace stack pointer (which is "sp").

So in summary, this part of FPU save/copy/restoration logic is triggered only upon usage of FPU.