I invoked undefined behaviour. Do I need a new computer?

Question

Suppose I'm working on a computer and write a C program which invokes undefined behaviour. Since anything may have occurred, a (perhaps paranoid view) would be that anything the computer does thereafter is no longer trustworthy. For example, it may have rewritten the compiler, or modified the clock. Is it possible to restore the computer to a trustworthy state? Do any data security standards require action after a (trusted) user invokes undefined behaviour (from known code)?

My understanding is that (a) it's not uncommon for developers to occasionally write such bugs and (b) no action is ever required in practice. But I have not worked in an environment where a very high level of trust is required.

My guess is that if any action is required it would depend on the following factors.

1. The sensitivity/value of the work being done on the computer (i.e. maybe worth it for top secret, or if the computer is going to space)
2. Whether the code which invoked UB is known or not (e.g, even if the code can be identified, but the actual inputs are lost), or what type of bug was actually invoked
3. The compiler and operating system used at the time.

And of course the cost of the action necessary to restore it a trustworthy state. Obviously I'm assuming that the computer was in a "sufficiently trustworthy" state beforehand.

Answer 1

The paranoia level should depend on:

• the underlying OS

  Modern system offer process isolation: even when a process goes fully mad, it can neither write into the kernel memory pages, nore in memory pages from other processes. Things can be different in embedded systems (or if good old MS/DOS is still being used): a process invoking UB can easily crash the system possibly in the middle of a file system operation and end with a no more bootable system.

• the evilness of the programmer. Most of use do not invoke UB too often, or do not try to use UB to circumvent system protections. The more you try that, the higher the risk of breaking the system

• the caution used during tests. Normally, test should only occur using unprivileged accounts limiting the possible damages to what is accessible to that user. But certain part of code do require admin authorizations, for example to access privileged network ports on Unix-like systems. The more code being run under admin account the higher the risk.

The damage level to be expected:

• application crash: high risk as soon as UB has been invoked. Just restart the app (if possible after fixing the error...)
• system crash: should not be directly possible, unless your run your tests under an privileged account. But a simple reboot should be enough.
• damages to application files: Once UB has been invoked, the application could continue to run and write garbage to application files. It can require a full re-install of the application
• damages to system files: should not be possible under you run your tests under a privileged account. But you could have to reinstall the whole system from scratch
• hardware damages: This one should be very uncommon. It should only occur if the system was tweaked for example with higher frequences than recommended by the hardware provider. The application could then enter a loop asking to much on the hardware and leading to too much heat and irreversible damages. Honestly I have no real world example of that

TL/DR: if you use a non priviledged account in your developements, then you should be prepared to reinstall only your dev environment. If you use an admin account, you should be prepared to re-install the whole system. But having to replace the hardware should only occur for very special use cases...

Answer 2

In Theory

According to the C reference, it is entirely valid for a compiler to install malware on your system, should it encounter undefined behavior, such as using memcpy() on overlapping memory areas. Such a compiler, while clearly being malicious in nature, would still be standard compliant. The resulting binary, when run, could also cause your hard drive to be erased or the disk to be encrypted by ransomware. Again, all of this would be standard compliant.

In Practice

In practice, "undefined behavior" usually means that something acts one way on one platform, and another way on another platform. For example, imagine the following memory area, where would like to copy 4 bytes from src to dst:

0 A 5 C B B 1 7 2 0 4 7
  ^-----^       ^-----^
  src           dst

The resulting memory will look like this:

0 A 5 C B B 1 7 A 5 C B

Now, imagine if these overlapped instead, as follows:

0 A 5 C B B 1 7 2 0 4 7
        src ^-----^
                ^-----^ dst

How would the resulting memory look like? 0 A 5 C B B 1 7 1 7 1 7? 0 A 5 C B B 1 7 1 7 2 0? The result will change, depending on how exactly memcpy() is implemented. I'm certain if you gave a class of students a task of implementing that function themselves and tried it on this example, you'd get different results with each implementation.

So instead of trying to define how an implementation has to act when encountering such a situation, the standard just says it's undefined. Anything goes. Though in practice, "anything goes" will mean that the compiler will just do it in a certain way and that's how it is. And if you rely on the specific way the compiler does it, and it changes in the future, that's not a bug, because you're not supposed to rely on undefined behavior.

---

An ongoing in-joke among C and C++ programmers is to refer to undefined behavior as "nasal demons", because "The C compiler can do anything, even make demons fly out of your nose." And this actually illustrates the issue very well: While it would be standard-compliant to make demons come out of your nose, it's also very unlikely that a compiler would actually do that. Most programmers want to write sensible code^{[citation needed]}, so it's very unlikely that someone will deliberately try to harm the security of your computer, merely because of undefined behavior. After all, they still wrote code to harm your computer, and that usually doesn't go too well for someone's reputation in the open-source community.