There is always a risk when connecting to a server with any client software. Suppose that the client (ie OpenSSH or PuTTY) holds data from the server in a buffer with poor bounds checking... it might then be possible for the server to remotely execute code on the client machine.
All software has errors, so it is not impossible to think that an exploitable vulnerability exists in your client. For this reason it is wise to be very careful connecting to machines that you actively distrust.
Sever software tends to be at a greater risk since anyone can try to attack it any time. Client software is still software though, and connecting outwards provides an opportunity for the server to attack the client.
Mitigations like using ASLR and the DEP on your SSH client and always keeping the client machine up to date on software patches are probably sufficient to prevent casual attacks. Disabling privacy eroding features like agent forwarding is pretty smart. X11 is disgustingly bloated and insecure... so do not use X forwarding under any circumstance.
PS. Regarding using a disposable middle man computer to connect to the VPS: maybe try using a different SSH client on each, like PuTTY to connect the Heroku app, then OpenSSH to the VPS. What are the odds both clients have an exploitable bug that the attacker knows about?
PSS. One of the answers to this question mentions a real attack on the OpenSSH cleint: Connecting to a compromised server