How can a Linux server be protected against an attacker that tries to open a large number of SSH sessions (probably trying to brute force a password) that no ports remain to be able to legitimately login to the machine?
The server already uses only SSH key based access and does not allow root logins, so the password brute forcing is not a concern.
One usually evoked solution is Fail2Ban: this is a system which uses the firewall rules (iptables) to block incoming connections from IP addresses from which some kind of exhaustive search attack is apparently in progress.
This, of course, won't work with a distributed DoS, coming from thousands of distinct IP addresses. In general, very few things resist a distributed DoS.
Fail2Ban can also be problematic in case of large-scale NAT: it has potential for locking out a whole network of normal users who just happen to share the same IP address, as seen from the outside.
It is highly recommended, when you begin to configure tools such as Fail2Ban, to also run another SSH server on another port, not covered by the Fail2Ban rules, just to allow re-entry in case a bad configuration locks you out.
Another completely different strategy is to keep a connection open at all times. E.g. you have a long-standing SSH between your server and your own desktop system; you may even multiplex other connections on it (with SSH abilities for tunnels, and/or SOCKS proxying). This avoids the allocation problem: you don't need to get a free port for your root connection since you already have one.
