Edit:
After looking at the page where this question has previously been addressed I would definitely recommend a read of the green-checked answer there. It outlines well most of both the positives and the negatives of changing the port, and suggests that whether or not doing so is a good idea depends on the circumstances. For the types of clients I most often have--small(ish) businesses with small networks, for the most part-- I will usually recommend or implement moving the SSH port away from the default on at least some machines (Net-facing, DMZ-facing, etc.). But other people in other IT circumstances might not even consider it as a serious option. Vive la difference.
I will say, though, that I like @Steffen Ullrich I get irritated by the "But it's Security by Obscurity!! It must be terrible!" point. Yes, it's Security by Obscurity. So what? If you're really using a defense-in-depth strategy Security by Obscurity can be a positive contributor, another layer that adds to the strength of an already robust defense. Should you use a measure that falls under Security by Obscurity as your sole or even a main line of defense? Barring some extreme, hypothetical circumstance, of course not. (For example, moving your SSH port is obviously no substitute at all for requiring strong authentication of anybody who does manage to connect to it.) But being able to deceive, slow down, frustrate, and confuse an attacker is a good thing too in InfoSec. Just as it's a good thing in war, poker, sports, or ... well, in pretty much any adversarial situation that ever occurs in human experience.
And finally, you know who uses Security by Obscurity? Everybody, everyday. Every security pro or admin who doesn't publish to the Internet a comprehensive, exact listing of every device, every physical connection, every OS, every application, every authorized user, every administrator, every piece of security software, and every other security-relevant detail for every network they work on relies on Security by Obscurity to slow down & hinder prospective attackers to some degree. It's just relying on Security by Obscurity alone that's profoundly stupid.
Anyway :) , original answer starts here:
Changing the SSH port is a practice I try to follow whenever I can and advise clients to follow where reasonably possible (ie. whenever that can be done without breaking access scenarios for elaborate, labyrinthine administration setups that evolved resting on the assumption that SSH would be on the default port). Why? Well:
- As others have said, it means that a typical, quick & dirty port scan run against large numbers of ip addresses by an attacker who's just looking for potentially-vulnerable common services on any machine he or she can find will usually miss detecting your SSH service. Couldn't a determined attacker who was specifically targeting your box and trying to find your SSH service just check every port? Sure. But for most bad guys doing something like that is only practical if you've already found & selected your target machine in the first place. In other words, a bad guy using a broad scan against many ip addresses to discover interesting targets, whether from outside your network perimeter or inside it, is (usually) not going able to (or want to) check hundreds or thousands, let alone tens of thousands, of ports per ip address. It simply isn't practical to do that--usually, for most attackers--across more than a few potential targets. Thus, changing the SSH port can sometimes meaningfully reduce the visibility of your machine against non-targeted attacks.
(Although, if you are indeed doing IP and/or MAC-based filtering properly non-targeted attacks should fail anyways. But for the sake of defense-in-depth it's still a desirable trait to have.)
- But okay, what about targeted attacks? What about the attacker who is willing to scan all 65000+ ports for your listening SSH service? Well, obviously an attacker like that will find what he or she is looking for, your SSH port. But here changing the port can still be helpful if you have an IDS/IDPS in place & configured skillfully. Why? Well, simply put scanning tens of thousands of ports on a machine is a much "higher profile" event than scanning a relative handful. The reason is obvious: very, very infrequently is a device receiving scans across tens of thousands of ports a sign of anything but a concerted attack. Even if the scans come from a number of different ips, stretched across hours or days, there's just no way to really hide it when an attacker needs to check thousands upon thousands of ports on a machine, especially that are rarely if ever used by common services. With any kind of thoughtful configuration/setup of an IDS (host-based or network-based) and appropriate monitoring of alerts you can create a nasty tripwire for an attacker who just says "Well, I know how to find SSH. I'll just scan everything on the box...".
Of course, if your attacker is a little more patient and has internal network-side access to your box he/she may be able to just sniff for SSH traffic and see what port it's directed at. But this gives your opponent one more opportunity to be impatient and screw up...
At the end of the day, changing the port should definitely be just an additional, supplementary factor of protection that an attacker has to deal with. It is not a substitute for doing the SSH basics right (strong factors of authentication, proper configuration of auth-try throttling/ lockouts, etc.) But it can still be a useful measure.