Generally, asymmetrically encrypted chat handshaking starts out with exchanging public keys. Ideally you should have some mechanism for tying the key to the owner (think SSL certs and signatures) to prevent MITM attacks. Easiest to just use an existing system (such as SSL certs or PGP keys) to avoid reinventing the wheel. What signatures you decide to trust depends on what sort of trust model you're interested in maintaining. A good place to start is to determine exactly what constitutes a user's identity, and what type of access an attacker would have to have to steal that identity.
Then, of course, use RSA to encrypt ad-hoc session keys used for symmetric crypto like AES, Twofish, RC4, etc. The efficiency lies in the fact that you're only using RSA to establish the identity of the party you're communicating with, and then you immediately switch to something more efficient.
On the other hand
If you're talking about using RSA in a broadcast medium (think IRC rather than Jabber), then yes, this is terribly inefficient. Encryption is necessarily point-to-point, rather than point-to-multipoint. Each stream is individually encrypted. So the number of encrypted streams streams a given client produces is equal to the number of destinations that message is sent to. Each one set up independently of the others. This is precisely why systems like IRC do not encrypt end-to-end between different clients. The overhead would be just too much. At the very most, they encrypt client-to-server, allowing each user to have a single secure channel, with the server handling the message distribution.
That's not to say that what you're doing is impossible or ill-advised. If you want to build it, then go ahead.