We are doing a proof of concept project for a client to introduce smart card authentication for his operations admins. Authentication in client's context means mainly SSH access. We encountered a problematic usage scenario: Admins often issue bulk commands against many hundreds of remote machines. This took previously insignificant amount of time, but takes tens of minutes with smart card.
I have run a few basic benchmarks and found out that smart card based cryptographic operation indeed takes hundred times more than for example openssl with on-disk keys. What takes openssl often less than a milisecond on average, smart card does rarely in less than 0.5 second. It still seems fast, but multiplied by hundreds it becomes a major part of the observed delay (together with other smart card/driver induced slow downs). I searched what speed is usual in smart cards or if there are other personal hardware cryptography modules designed for speed, but it seems to me that for personal usage the speed of 2048 bit RSA based algorithms is never less than several hundreds of miliseconds.
What is the good architecture for large scale hardware token usage? We can setup a key distribution server for short lived keys which could be added to ssh agent, so that only authentication to such server would be secured by smart card and other connection would be secured by provided software key, but I consider it the least attractive possibility.