I would like to know how I can know which is the safest compilation line, that is:

Having several compilation lines in, for example, GCC, how do I know which one is more secure? Hardening would be a good solution? What do you recommend?

Does the compilation optimizations flags affect security?

  • 1
    What do you mean by "compilation line"? Do you mean compiler settings?
    – user163495
    Mar 10, 2021 at 17:12
  • 1
    @MechMK1 I think the OP is talking about which command line switches would lead to behaviour in these situations or similar.
    – user
    Mar 10, 2021 at 17:41
  • @MechMK1 I mean how to compare the security of two builds with certain optimization flags. Know which is better and that it does not generate problems as the anonymous user comments
    – sgio
    Mar 10, 2021 at 21:15
  • @sgio You'd need to output it to assembly (-S for gcc I think?) and compare them to check which ones remove the security features in the code. Or load them into NSA Ghidra or similar tools after compiling.
    – user
    Mar 10, 2021 at 21:21
  • This is a trust problem How do i know a compiler isn't injecting malware into my program
    – kelalaka
    Mar 10, 2021 at 21:26

1 Answer 1


Compiler options can influence the security of the resulting program in several different ways. Generally speaking, optimization can hurt security. However, this is not a reason to always turn off optimizations! For example, if your encryption code is so slow that users disable the encryption feature and send data around in cleartext, that makes security worse.

Undefined behavior

Low-level languages such as C and C++ have a lot of areas with undefined behavior. "Undefined behavior" means that anything can happen, and compilers take advantage of this to optimize code.

There a strong disconnect between the way typical developers see code and the way compilers see code. Often, what looks like an obvious optimization to a developer is hard for a compiler to detect, while compilers may perform optimizations that seem to the developer like it's exploiting a loophole.

A classic example is that optimizing compilers will eliminate redundant checks. Sounds good, right?

if (config == NULL) return ERROR_BAD_CONFIGURATION;
… // code that doesn't modify config
if (config != NULL) x = config->value;

An optimizing compiler is likely to compile that last line as if it had read x = config->value since it knows that config cannot be null at this point. Now consider this:

int *p = &config->value;
if (config == NULL) return ERROR_BAD_CONFIGURATION;
x = *p;

config->value promises that config is non-null at this point. Therefore the check config == NULL below is redundant and an optimizing compiler is likely to remove it. So, if config is null at runtime, x gets set to a value that's read from an invalid address, leading to a crash or worse. This is an example where an optimizing compiler makes incorrect code insecure, whereas a non-optimizing compiler would let the program get away with being incorrect.

Turning off optimization usually leads to fewer such surprising effects, but it is not a guarantee. The fix for such "optimization-induced bugs" is to write correct code in the first place, rather than hope that the compiler will figure out the intent of the programmer.

Undefined behavior is not limited to low-level languages. In particular, regardless of the language, concurrency inherently has some undefined behavior, because the order in which things happen is not determined. Race conditions can happen in any language (even without built-in support for concurrency, it can happen via communication between programs). Compiler options usually have little influence there.

Side channels

Some vulnerabilities exploit direct ways of accessing information, for example a missing permission check or a buffer overread. Other vulnerabilities are due to indirect ways of accessing information: side channels. A common side channel is timing, including the timing of indirect events.

For example, on a typical high-end system such as a personal computer or smartphone, the operating system protects programs from accessing each others' memory. However a program may still be able to deduce information from the way other programs behave. In particular, by measuring the timing of memory accesses, an attacker program can deduce which cache lines a victim program is accessing, and this provides information about which addresses the victim accesses in memory. For example, the easy and fast way to implement the popular cryptographic algorithm AES uses look-up tables; if the attacker knows which row of the table the victim is accessing, that can allow the attacker to reconstruct the key.

Programs can be written in a defensive way to avoid leaking information in such ways. This is sometimes difficult when optimizing compilers turn code without obvious leaks into executable code that does have leaks, because the compiler has determined that the program would be faster that way. In this case, an optimizing compiler can make even correct code insecure.

Compiler bugs

Compiler bugs are uncommon, much less common than bugs in some random application. Any decent developer knows that if the program doesn't work, a bug in their own code is much, much, much more likely than a compiler bug. Nonetheless, compiler bugs can happen. The higher the optimization level, the higher the risk of bugs. Most code doesn't benefit from the highest optimization levels anyway: usually -O or -O2 is a good compromise between performance and risk.

Safety features

Some safety features are partly under the control of the compiler. There, the obvious thing to look for is whether these safety features are enabled. These safety features generally don't help to defend against vulnerabilities, but they can make exploits harder. They often don't make exploits completely impossible, but they can make a difference between taking 5 minutes and 5 weeks to craft an exploit, and that can give the developer the time to develop and deploy a patch.

For example, many compilers can generate stack canaries. Stack canaries detect when a buffer overflow happens on the stack and halt the program. To exploit a stack buffer overflow, the attacker needs to limit the size of the overflow so that the canary isn't overwritten or arrange for the correct value to be written in the canary's place, neither of which are always possible or easy.

  • I really appreciate the time you have dedicated to answering me and your answer helps me a lot, exactly that is the problem, or the problems, that I want to detect using a tool that allows me to analyze the possible compilations, this is what I cannot find and I have a time searching, thanks.
    – sgio
    Mar 11, 2021 at 18:40
  • Does there any compiler that really eliminates that code? I would rather say, that is a really bad scope for the compiler. I've always said that a good unit test is required for all cases. Of course, perfect programming ( not even talking about constant time) is hard, so there are always bugs around. Also, I still believe trust is the core. This time solorwind, next time Intel C/C++?
    – kelalaka
    Mar 11, 2021 at 18:41
  • @sgio Unit tests it the key. Compilers are complex programs. If you want to become a good programmer in one, you should start to read the compiler internals, too.
    – kelalaka
    Mar 11, 2021 at 18:44
  • @kelalaka That example is a classic. It's perfectly normal from a compiler's perspective. It's mentioned in a blog post that user shared in a comment on the question](redhat.com/en/blog/security-flaws-caused-compiler-optimizations). Backdoor only account for a tiny fraction of vulnerabilities, so if your view of security is limited to trust, you won't get far. Almost all vulnerabilities are due to mistakes. Mar 11, 2021 at 18:46
  • @kelalaka Unit tests are not nearly enough. They won't find cases that the programmer hasn't thought of. For that you need more thorough approaches such as static analysis and fuzzing. Mar 11, 2021 at 18:47

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .