I need to process data that is sent by users in a LargeNumber format. My current choices are

  • BigInteger (.Net Managed Code)

  • IntX (Managed Code, faster for some operations)

  • GMP / MPIR (Unmanaged code, fastest with a wrapper 1 or 2)

Since dealing with large numbers has the ability to create some floating point errors, and the unmanaged code has the ability to create buffer overflows:

  • Is there anything I can do to proactively scan and validate large numbers that will be used in these libraries?

  • Are certain operations more risky than others? (mult vs add)

  • Are some libraries better than others? (Is managed code more secure because of bounds checking? .. does it even matter, it's just a number?)

  • Is it even possible to scan a large number for unusual cases that might cause an overflow? What would I look for?


A number is a number. When dealing with numbers, there are three steps:

  1. Decode the values from an external representation (e.g. a bunch of bytes which, in your protocol, follow a given convention).
  2. Apply mathematical operations on the values.
  3. Encode the results.

Buffer overflows due to malicious data will occur only in phase 1 and 3, and then quite rarely. During phase 2, there's no issue of that kind (if there is an overflow, then that's a bug in the library itself, but no such bugs is known for existing implementations -- as a rule, developers for mathematical libraries are maniac about tracking and destroying bugs).

As a user of the library, you must take care of the two following things:

  • When encoding, do not assume that the value "will fit in a given buffer length". Use the library to see the actual length of the value.
  • When using a lot of multiplications, integers can grow to unbounded sizes, leading to huge memory allocations and heavy CPU consumption. You may want to include checks on the length of integers at various places in your code, to see whether you are dealing with values which are way larger than they should.

I suggest you begin with BigInteger (from .NET) because that's the simplest and it is already there. Envision switching to something else only if you encounter an actual performance issue, duly measured and quantified.

(You will not have any "floating point error" when using integers, by definition. Floating point is quite another domain.)


As others have said, reading/printing is your major risk area for buffer overflows.

However, it is possible you could use the bignum library to spray the heap. For example, MPIR/GMP use a bignum structure which looks a little like this:

typedef struct
    off_t count;
    mpn_limb_t* limbs;
} mpz+t;

Where off_t describes the number of limbs and its sign describes the sign of the overall number. Then each mpn_limb_t is typedef'd to an appropriate size for your platform.

Clearly, this cannot be and is not stack allocated, so you're basically writing all of these numbers (mpn_limb_t) to the heap, allowing you to heap spray which won't directly exploit the library, but it may facilitate another exploit.

The sanitization process for this is to not trust a binary serialization of your number, since every bit is used to store the number in raw form and put that into memory that gives you the full range of opcodes. You really need to read the number from a human-readable format, which whilst less efficient, does make it somewhat more difficult to produce useful shellcode (unless you can write something with the opcodes represented by the ascii of 0-9, which I doubt).

Addition, multiplication etc "cannot" overflow, in the sense that gmp/mpir at least will fail to allocate sufficient memory for the target mpz_t and stop there.

There should be (there definitely is a print function) a function for reading integers from text representation. I would ensure you use this, and use the corresponding print function for output. You will then be able to handle numbers up to the limits imposed by your memory while not writing arbitrary nonsense to memory. You may wish to check the documentation for these functions to check their behaviour on non-integer/decimal characters.

Floating point exceptions shouldn't be caused by any mp float library, because the internal size/representation of the float in question can't be handled by the processor anyway. If you have a look at the algorithms (see algorithms for APFPA used in this kind of code, you'll see they use normal precision floating point operations. Any decent library really should ensure they do not cause a floating point exception when working on "limbs".

  • Does this FhtMultiplicationException make sense to you? (ctrl-f for the exception) – goodguys_activate Mar 5 '13 at 13:15
  • @maker it sounds like he's saying if he detects precision loss, then he throws that exception. I think that's his exception, rather than a FPE from the CPU. If I've understood it correctly anyhow. I could be wrong. – user2213 Mar 5 '13 at 14:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.