Conceptual view: there is authentication, and there is authorization; these are distinct activities. Authentication is about making sure of who you are talking to; authorization is about deciding what some individual is allowed to do. You actually want to keep them separate.
RFID tags implement authentication: through the electronic conversation between the tag and the reader, the reader ascertains the identity of the tag, so (presumably) the identity of the tag holder. Authorization is performed by the reader, that can be linked with some central authorization server. On the other hand, with a key, authentication and authorization are conflated in the same device: having the key in hand automatically grants access.
The need for separation of authentication and authorization is made most visible when you want to revoke an access. With RFID tags, that's easy: simply flip some flag in the authorization server database. With keys, you cannot do that: to revoke an access, you must either recover the key itself (and you cannot do that if the reason for revocation was that the key has been lost or stolen), or change the lock (and that's expensive, not only for the lock itself, but because you must distribute a new set of keys to the other users). In a similar vein, with RFID tags you can enforce time-based access control (access granted only at some times in the day), and you cannot do that with keys.
Another good property of RFID tags is selective cloning. When a new employee must be granted access, issuing a new tag for him is easy for the system administrator; but users themselves will find it hard to clone their tag (good tags are like smart cards: they are tamper resistant). This means that authorization management remains in the hands of the sysadmins. With keys, either keys cannot be duplicated, in which case granting access to new users is hard, or keys can be duplicated, in which case preventing rogue cloning by existing users is hard.
Summary: RFID tags are superior to keys because they allow for fine-grained access control with immediate revocation, and centralized authorization and administration.
The most salient point which would make keys preferable over RFID tags, in some specific contexts. is that keys don't need to be powered. A padlock which has been locked, remains locked even after an extended power outage. However, in most cases, RFID tags are better.