Floating point numbers are approximation of real numbers. They are appropriate for continuous computations, where a small variation of the input results in a small variation of the output.
Cryptography generally involves computations that are as far as it gets from continuous. No matter how similar two non-identical messages are, they should not have similar hashes, or similar signatures, or similar encryptions. Floating point are wholly inappropriate for this.
Cryptography often involves some kind of irreversible-looking scrambling, which you can only reverse given a secret key (or not at all). Natural processes tend to be continuous, hence reversible. Chaotic processes, whose past cannot be computed, tend to involve integers at some point — often a number of iterations or loops, where a threshold is hit or missed at each iteration. Integers are the way the irreversibility arises.
The result of cryptographic operations has to be reproducible. For example, the decryption process has to exactly reverse the encryption process. Floating point numbers are at a disadvantage there because operations on them tend to have variations among different processor architectures.
A lot of cryptography, including RSA, relies on “nice” properties of integers, often related to divisibility (which is fundamentally a property of integers). Floating point numbers have no such properties that make the algorithms work.
Complexity is not what makes cryptographic algorithms secure. It's a combination of mathematical properties, and of nobody having found a way to break them. There is absolutely nothing to be gained with floating point computations there, and as we've seen a lot to lose.