3

I have a web app hosted in Digital Ocean and use Laravel Forge to keep a daemon running an SSH tunnel to another server (i.e. ssh -L XXX:127.0.0.1:XXX -p XXXX root@[ipaddress]).

I do this in order to stay connected to a remote DB in that server.

Is there any major vulnerability in doing this? Can someone else see the daemon command running? But even if that was possible, wouldn't they need the SSH key to have access to the server?

4

Is there any major vulnerability in doing this?

Depends on the configuration / threat model / etc. There's a lot of variables to consider.

Can someone else see the daemon command running?

Yes - anyone with access to either machine can see this, including all parameters.

Linux:

ps -aux | grep -i 'ssh'

macOS:

ps -Aj | grep -i 'ssh'

Here's an example:

$ ps -aux | grep -i 'ssh'
root      444  0.0  0.0  37588  5692 ?        Ss   13:20   0:00
/usr/local/bin/ssh -L XXX:127.0.0.1:XXX -p XXXX root@[ipaddress] 

But even if that was possible, wouldn't they need the SSH key to have access to the server?

Well, yes and no. There are some factors to consider:

  • If someone is able to compromise your server for any reason, you may cause the compromise of the remote database server depending on the configuration and permissions.

  • If the SSH server allows password-based authentication, that could present a problem if they discover credentials on your system.

  • If your SSH server accepts keys, but your key doesn't require a passphrase, this can lead to compromise of the database server.

    • You should ensure a significantly-strong passphrase exists to use your SSH key (better practice) to prevent an attacker from immediately compromising the database server after grabbing your key.
  • If an attacker compromises the database server, they may be able to compromise other servers in the network. You may be opening up a huge can of worms here.


Defense in Depth

As with everything security, it's important to consider your threat model and layering. Defense in depth is the process of creating a bunch of different layers of security to prevent as much compromise as possible.

Ask yourself, "What do I have, and what am I trying to protect?" This is the starting point for security engineering.

In this scenario I would recommend something like this (assuming you're the DB owner):

  1. Ensure that the web applications have sufficient hardening to prevent remote code execution, sql injection, ssrf, lfi, rfi and other vulnerabilities.
  2. Ensure that the web server exposes only necessary ports such as 80/443.

  3. Ensure that the database server itself communicates only with your web app via SSH. By default, it's going to deny everything except what is required to run, and allow your web app's IP address on port 22. Depending on the scale and complexity, you may have it communicate only with the load balancer which will distribute it across other web app instances, but that's out of scope here.

  4. Ensure that your SSH key requires a passphrase. A very strong one.

  5. Ensure that the database does not have file I/O capabilities unless necessary.

  6. Ensure that the database server cannot, by default, create any outgoing connections except on required ports. This ensures that if someone exploits your database through some kind of injection or exploit (Attacker -> Web App -> SSH Request Forward -> DB Exploit), it doesn't allow the attacker to (easily) return a reverse shell to their system (we covered SQLi in #1, but still -- here's another annoying layer).

  7. Ensure that neither the web app server, nor the database is running as root.

  8. Ensure that the permissions for your SSH pivot are as low as they can be to manage it. You may set up users in the database configuration for example.

  9. If possible, ensure that the database server is on it's own VPC/VLAN.

  10. Finally, if you really do need to go this route, there should be no reason for you to connect as root. You should connect as a standard user and type in a strong password. A VPN would be a better idea.

This is not a complete list, just something quick to get you started.

TLDR: principle of least privilege.

| improve this answer | |
  • Thanks for such detail, I'll carefully review everything here. It turns out the database server was compromised, but "attackers" only turned off apache service. I really doubt this could have happened because of that tunnel running in my DO+Forge server. I'm thinking it was rather due to poor configuration on the database server, this is not even a commercial application. Anyway, this is some great info to look at. – Jk33 Aug 13 '18 at 19:51
3

First glaring issue: ssh root@

Don't do that. Don't ssh as root. Ever. SSH as root is bad practice, is worse than sudo su -. If your password or key is compromised somehow, the entire DB server is gone. Privilege separation, tight permissions, full disk encryption... all useless.

If someone gets a shell on the application server, he can connect to the database service (service, not server) without needing the SSH keys. Why? It's a TCP tunnel! They can connect straight to the database using the local port. They don't even need to find you are running a tunnel, just looking at the open ports with netstat is enough. Or looking at your config files. And what else will be at the config files? The database credentials.

If you SSH keys aren't password-protected, they can access the entire server, as root. They already had the database credentials, now hey have root access too.

I would use Wireguard or another VPN service to tunnel requests, and configure the firewall on the database server to only allow connections from the VPN to the database port.

And I would change SSH default port 22 to something else, and put a honeypot on port 22, something like SSH-Pot: An ssh server that never authenticates. That makes random scan bots to miss your server.

And as Mark said, hardening and least privilege. Disable as much as you can. If you don't need email, disable email. Disable every and any service you can, until the service stops working. Configure the firewall on the database server to not allow any incoming connection, except the VPN service itself and the DB port inside the VPN, and not allow outgoing connections. Do the same on the application server: allow ports 22, 80, 443, and the new ssh port (you changed it, right?), and don't allow outgoing connections except the VPN service on the DB server, and the DB service.

| improve this answer | |
  • Is there a way to keep an SSH tunnel open if the key had a passphrase? Or actually, is there a better way to remotely connect to a database without needing to keep the SSH tunnel open? – Jk33 Aug 13 '18 at 19:54
  • 1
    To remotely connect to the database, use a VPN. That's what VPNs are good for. – ThoriumBR Aug 13 '18 at 20:09
  • ^ This. Use a VPN. I was just trying to work with your current setup (which I do not recommend). – Mark Buffalo Aug 13 '18 at 22:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.