Improved CPU speeds will technically help, but probably not by as much as you hope or want. Especially nowadays, as most performance gains are made by adding cores to chips, rather than increasing clock cycles.
Operations in asymmetric encryption are typically much, much slower than their counterparts in symmetric encryption. For this reason, asymmetric ciphers are frequently used for the purpose of securely exchanging secret symmetric keys, which can then be used by both sides to encrypt and decrypt data in bulk. You may want to investigate this possibility.
Another option is to run the decryption tasks in parallel. You probably have at least four cores on your server, but a typical PHP page will only use one of them to render it. If those decryptions are completely independent of one-another, they can be run on multiple cores simultaneously, giving you a sizable performance boost for "free".