4

In order to avoid problems with integer overflow in C or C++, some people have suggested to use unsigned integers. How can this protect against possible overflow attacks? Doesn't an unsigned integer still eventually overflow when incremented beyond its maximum value?

9

Using unsigned integers does not prevent overflow. A k-bit variable can only represent 2k different values. No matter how you interpret the bits.

What you're probably referring to is the fact that signed integer overflow in C and C++ is undefined behavior. Which means that the C or C++ international standard does not specify any behavior of the program that triggers such overflow. Wrapping the value around or causing a run-time exception are valid options as much as opening an exploit shell. (Of course, no sane implementation would opt for the latter choice. But an exploit can become possible through further consequences.) Furthermore, the compiler is allowed to optimize based on the assumption that signed overflow does not occur and might, for example, optimize away misguided overflow checks such as if (x + 3 < x).

For unsigned integers, on the other hand, the C and C++ international standards require that the value wraps around as if arithmetic on a k-bit unsigned integer was done modulo 2k. This means that your program can rely on the wraparound and use it to its benefit. (Or to shoot itself into the foot.)

Other languages like Java mandate reliable wraparound also for signed integers. (Java doesn't even have unsigned integer types.)

Speaking of security, the largest threat posed by overflow, however, is that your application logic does not anticipate the wraparound. And this is a problem regardless of whether you are using signed or unsigned integer types. If your application logic assumes that for a request of n bytes, it needs to allocate a buffer of n + m bytes (for a small constant m used for, say, bookkeeping) and n happens to be less than m below the maximum value your integer can hold, wraparound will cause your program to allocate a too small buffer which, in turn, will be overrun. There goes the exploit.

If you want to be secure, instead of using unsigned integer arithmetic throughout, I recommend that you do the exact opposite: Use signed integer arithmetic and compile with UbSan. Unless your application is computation-heavy, it will have only minor run-time overhead but if the unanticipated event happens and your application overflows an integer, extra code added by the sanitizer will kill your application before an attacker can exploit it. A similar sanitizer for overflowing unsigned integer types cannot be written because the standard requires well-defined behavior for doing so and programs legitimately rely on it. This is actually one case where undefined behavior in the language can help make your application more secure.

|improve this answer|||||

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.