I just realized that I need SSH agent forwarding in order to push to one Git server that then forwards the commits to Github.com. I never used SSH agent forwarding and don't understand the security implications of enabling this feature. Do I have to trust the remote server in order to allow SSH agent forwarding to it?
Yes, you have to trust the remote server if you allow agent forwarding for this server (especially if you use the same key for more services). Doing so rogue root or evil admin with root access can impersonate you for authentication to other servers during the time you are connected to that server.
The state is still better than accessing the key permanently or stealing it from you, what could happen when you would copy your private key to the server, but this is still an issue, since there is no confirmation on your side, that you really want to do crypto with your key in agent.
Using separate key for github is probably good idea. And having middle-step-clone of your important repository on a server you don't trust is probably something you also don't want.
This question is a few years old and things have changed.
In the default configuration, ssh agent forwarding is still a security issue.
PuTTY and OpenSSH have added a lot of features regarding agent forwarding and depending on the client, the configuration and the operating system, agent forwarding is more secure, than using a password protected private key on the remote server.
This combination only works on Linux machines! (see notes about Windows, MacOS not tested).
If agent forwarding is used, the private keys must be protected with ssh-askpass or a fido2 token.
The recommended method is using PuTTY in combination with the OpenSSH agent. PuTTY is able to detect and mitigate spoofing attacks in recent versions.
When using PuTTY>=0.76, you should use Disable "trivial" authentication (SSH-2 only). This option can be found under "Connection -> SSH -> Auth". In older versions (>0.71) you can use the trust sigils to detect spoofing attacks.
Pageant (PuTTY's agent) should not be used, because it does not support fido2 tokens or ssh-askpass. This is where the OpenSSH agent comes in.
You only have to start the ssh agent (if it is not already started). When PuTTY is started, it can use the OpenSSH agent without further configuration.
With this setup, you have to confirm each usage of the private key and abusing or spoofing a forwarded agent is not possible.
Since Windows 10 it's possible to install the OpenSSH client as an extra, but this version is not compatible with PuTTY. The reason is, the Windows version uses named pipes and PuTTY only supports sockets.