If you only have one client and one server and you are fully in control of both of them, it doesn't really matter where you generate the keypair. But if any of those assumptions are false, then it starts to matter.
I believe the "best" way to use your SSH keypair is to generate one keypair per client, and push that keypair's public key to all the servers that client is allowed to access. In other words, the client is the "trusted" computer and the servers are the leaf nodes the trusted computer is allowed to access (e.g. if you have a laptop and a desktop and a tablet, and each of those are allowed access to some servers, each of those servers will have 1-3 keys in their
.authorized_keys files, indicating which of your laptop, desktop and tablet are allowed to access them; this allows you to revoke access (by deleting lines from
.authorized_keys) if you lose control over one of the clients).
But if you generated your keypair on a leaf node, you are allowing any spyware (or anyone with access to your files) on that leaf node to gain access to all the other leaf nodes.
and you delete the private key from the server after acquiring it
What makes you think there isn't a process on the server that will store a copy of that private key before you can delete it? I'm not trying to pick on you, but you're making assumptions ("no one will look at files I create until after I log out"; "files I create won't be logged and won't persist after I delete them"; "passwords I enter won't be logged") that may not be accurate.