I always thought that the reason we couldn't use it was because it required too many CPU cycle. But an answer from @TomLeek indicated that this was not the case :
Most texts that talk about hybrid encryption say that we need to do that because asymmetric encryption is slow, but that's wrong. The real reason why hybrid encryption is used is because known asymmetric encryption algorithms cannot simply process messages of arbitrary length, and we have no real idea about how we could alter them in order to do so securely. Basically, the "chaining modes" for block ciphers do not have equivalents for asymmetric encryption that would be obviously safe.
Why is it so complicated to use RSA as a block cipher?
When you look at it quickly, RSA encrypt a block of text so we could call it a "block cipher" and then you just need to plug that into a good block cipher mode of operation and you would call it a day. BUT, it seems that RSA require some fancy padding to be secure. Would the security problem related to that padding issue?