I wonder if it is possible to write a kernel that would keep all of its RAM encrypted, storing the key in the CPU cache, so that the machine would be resistant to cold-boot attacks?

  • 2
    Possible duplicate of this question: security.stackexchange.com/questions/8278/os-with-encrypted-ram – Xander Feb 20 '13 at 23:28
  • 2
    Not really, I think. – d33tah Feb 20 '13 at 23:32
  • PrivateCore's vCage is designed around the assumption that everything but the CPU is untrusted, so that includes the RAM. The kernel itself is locked to the CPU cache, and the entirety of the RAM is encrypted, so the most the RAM could do, if it were maliciously modified, is cause the system to crash. The same applies to malicious PCI devices, the motherboard going haywire, etc. – forest Apr 8 '16 at 1:13

Data in L1 cache will not remain in L1 cache only; the hardware will copy it to main RAM transparently and almost immediately. At least so operate modern CPU. If you want to keep sensitive data out of RAM, then you must keep it in registers only. Context switches will be a problem, also, since they automatically flush registers to a designated RAM space.

While it is theoretically possible to operate only in registers and do encryption and decryption in the CPU only, it will be hard to do correctly (random-access encryption with updates is difficult to design without weaknesses) and it will be very expensive (you would more or less have to implement a virtual machine, where each opcode entails some decryption, so expect a slowdown of a factor 50 at least).

The TRESOR project appears to be much more limited in scope; it does not want to protect generic data from leaking to RAM, only encryption keys. Which, in my opinion, misses the point. You encrypt data because that data is sensitive and should be kept confidential. The encryption key is a high-value target because it concentrates secrecy, but certainly not the only target.

  • How about allocating as much RAM as possible and putting an encrypted swapfile then? – d33tah Feb 21 '13 at 13:09
  • Question is about not letting clear content make it to the RAM itself, not to the swapfile. Cold boot attacks plunder the contents of the RAM chips directly. – Tom Leek Feb 21 '13 at 15:20
  • Well you could use CAR mode (Cache As RAM) to ensure data in the cache stays in the cache (in this case the LLC, not L1), which is the solution used by vCage. – forest Nov 29 '17 at 8:06

... keep all of [the] RAM encrypted, storing the key in the CPU[?]

Yes, you could write a kernel that stored the key in a CPU register. SSE processors (basically any running Intel architecture) have 128 bit "XMM" registers that the key can be stored in, and the latest Intel chips can do hardware-based AES. One register, one key, one CPU instruction.

You'd lose a multipurpose register and have to swap into kernel mode very often (massive performance killer) or give each application its own key that is encrypted with the kernel's key, but it'd work. The wicked catch is that a context switch usually puts the register values into RAM. The key and anything else in the kernel have to be bumped because any application can use all general-purpose registers.

What's really needed is a persistent register that is addressable only by ring 0. I know these exist as control registers, but wasn't sure of any general purpose ones are laying around for the taking. I then looked at d33tah's link to TRESOR and it turns out somebody actually implemented it. Do read up on the caveats and such.

  • You could also use the x87 FPU registers which tend to be mostly unused on modern systems (now that SSE is a thing). Userspace attempts to do floating point operations using these legacy registers could be trapped by the kernel and emulated, which should be uncommon enough that performance is not an issue. TRESOR specifically uses the debug registers which can be disabled without even affecting the behavior of debuggers (as they use the more versatile software breakpoints now). – forest Jan 28 '18 at 21:45

Some Googling pointed me to http://en.m.wikipedia.org/wiki/TRESOR - using it with an encrypted swapfile sounds like a solution.

  • what encrypted swap did you go with? is the swap a ram drive? what do you think of cryptkeeper? – user24408 May 30 '13 at 4:23
  • CryptKeeper, if you mean the kernel patch which encrypts memory, only works for very old kernels. I don't know if anyone ported it to newer kernels yet, but it'd probably be a pain. – forest Apr 8 '16 at 1:15
  • TRESOR only hides encryption keys. It does not encrypt actual memory. – forest Nov 29 '17 at 8:05
  • 1
    Well firstly, swapping does not occur to kernel memory, so secrets in the kernel (page cache, various slabs, etc) cannot be swapped. Secondly, swapping is a very slow process, even if the swapfile is a ramdisk. Normally when a page fault occurs, a fastpath is taken which very rapidly prepares a page for the process. With swapping, a page fault incurs a massive overhead as a lot of kernel code has to be run. I tested this with a Debian live install, and found that even at 500 MiB of memory, and the rest in a swapfile in tmpfs, the system was barely usable. – forest Dec 1 '17 at 10:42
  • 1
    There is a kernel patch called RAMCrypt, which uses TRESOR internally and encrypts individual processes, but it only works on processes and not kernel memory, leaving only 4 KiB unencrypted, and it is quite slow. There is also a hypervisor-based solution, HyperCrypt, which does encrypt all but 4 MiB of memory, but it is even slower, especially for certain workloads common for desktops. – forest Dec 1 '17 at 10:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.