There are two main ways for an attacker to obtain the private key file:
- By compromising the client machine, in which case the attacker can usually obtain the password as well.
- By compromising a backup of the client machine, or stealing that machine.
If you're comparing private key authentication with password authentication, only the second case is relevant. A strong password on the private key file is indeed necessary for security in the second case.
You can reduce the risk by excluding the private key file from backups, but it sounds like you have little control over the client machines, and theft is always a possibility. You can also reduce the risk by having a way to deactivate a key quickly. If a penetrating your network requires a VPN connection in addition to the SSH key, this increases the time window for deactivating the key before any harm can be done.
Note that a password is not a panacea either. SSH clients tend to make password authentication difficult to automate, but sufficiently annoyed users will write the password in a plain text file anyway, and may even expose it in a script or configuration file somewhere. Using a private key file has the advantage that the confidential information is concentrated in one file that users don't tend to copy around.
Another consideration that is to the advantage of private keys is that they tend to be easier to revoke, from a social point of view. Telling a user that they have to change their password is never appreciated, and you have no way to make sure that they won't change from Password123
to Password456
. (Unless you force the passwords, in which case you know they've written it down somewhere.)
Furthermore, if the VPN has the same password as SSH, then for most intents SSH does not perform any extra authentication. Therefore SSH with key authentication represents an extra layer of security in this scenario (regardless of whether the key is well-protected), whereas SSH with the same password does not. If the alternative is having the password get you through the back door as well as the front door, having the backdoor protected by a different mechanism is a plus.