I have Raspberry PI 3 which has 4G modem dongle attached to it. It works autonomously and every one in a while has to connect to my server via ssh. This is done by service script either with password or with key authentication which both require password/passphrase to be present in a script or in a file. The problem is that anyone can take out RPi SD card and take that password/passphrase or id_rsa key itself. Are there any secure solutions for this?
The most secure solution is to have SSH keys encrypted with a passphrase and managed with a key agent (e.g. ssh-agent). However,
if your system reboots, it requires human intervention to reload the keys, making it inappropriate for some uses where automatic system recovery is needed. If you find yourself trying to think of ways to automatically load the keys into the agent, then you don't really want to use the agent.
(from the O'Reilly definitive guide to SSH at http://www.snailbook.com/faq/no-passphrase.auto.html)
If you must use a key without a passphrase, the guide says to be sure the key resides on a local disk (the SD card in this case), and you should also use restrictions on the public key such as "from=" and "command=" so that theft of the key (or the SD card on which it resides) can be mitigated.
If you want the Pi to be able to turn on and start working automatically then no, you can't secure the key. If it's able to use the key without your input, then the attacker will be able to as well (barring some sort of tamper proof hardware).
I think your best bet is to create an account on your server for the Pi and then restrict it as much as you can, that way if someone gets the key they won't be able to do as much damage before you revoke it.
Better yet, don't use ssh at all. You don't go into much detail, but maybe this is an XY problem. Perhaps you could keep a script with the commands you want to run on the server, and trigger it somehow from the Pi without letting the Pi actually run any commands itself. I've seen, for instance, a cron job using curl to post a secret value to a url to cause the server to run a script. If someone gets this secret value then they can run the script whenever they want, but they won't be able to do anything else with it.
- Encrypt the OS. and
- Setup your server to use Public Key Authentication with SSH. I.e. the server has a public key authorized_keys that will only accept connection from matching private key ssh-client. In fact, you should not store the id_rsa that pair with authorized_keys in the server.
AND Lock your RPI3 in a sturdy case.