Although public keys and authorized_keys files contain the counterpart of your private key you have good reasons to store them in a safe place. Read-only on your workstation is a start.
By convention, and openssh defaults, the authorized_keys file is stored in ~/.ssh of the account that you (and possibly others) log onto.
Access to that file delegates management of it to the people authorized by their key in that file. (In the absense of selinux) They could be tampering with the contents of the home directory. Therefore that location is a risk, but this risk can be mitigated with an option in
The files should be readable by user, file attributes can avoid tampering even further (chattr +i file)