0

Since a couple of days ago my Debian server started to get login attempts (see log file below). /var/log/auth.log goes only three days back, although the server has been running for much longer.

I noticed that the network in-traffic has jumped by 10x and the server stopped to accomplish it's work (scrapping and analysing weather data each x minutes).

Are these hacking attempts, or, just someone who has committed an honest mistake and is trying to log-in into the wrong server?

I re-started the server and got a new IP. The login-in attempts stopped and the server could re-start to work as expected. Is that enough?

Invalid users:

Feb 25 07:05:47 <my_server_name> sshd[20223]: Invalid user pi from 84.1.34.55 port 42168
Feb 25 07:05:47 <my_server_name> sshd[20225]: Invalid user pi from 84.1.34.55 port 42170
Feb 25 07:11:16 <my_server_name> sshd[20249]: Invalid user admin from 110.77.173.11 port 59058
Feb 25 09:26:11 <my_server_name> sshd[20693]: Invalid user cacti from 91.206.4.250 port 51831
Feb 25 09:29:56 <my_server_name> sshd[20699]: Invalid user system from 91.206.4.250 port 46048
Feb 25 09:34:07 <my_server_name> sshd[20720]: Invalid user oracle4 from 91.206.4.250 port 40576
Feb 25 09:38:17 <my_server_name> sshd[20738]: Invalid user kang from 91.206.4.250 port 35145
Feb 25 09:43:07 <my_server_name> sshd[20757]: Invalid user scaner from 91.206.4.250 port 58343
Feb 25 10:45:32 <my_server_name> sshd[20972]: Invalid user  0101 from 5.101.40.10 port 60675
Feb 25 13:01:14 <my_server_name> sshd[21480]: Invalid user packer from 178.238.227.236 port 48256
Feb 25 13:57:14 <my_server_name> sshd[21708]: Invalid user customer from 45.77.20.111 port 61447
Feb 25 19:26:16 <my_server_name> sshd[23390]: Invalid user admin from 41.238.155.19 port 54046
Feb 25 19:26:21 <my_server_name> sshd[23394]: Invalid user admin from 123.17.142.134 port 8115
Feb 25 19:26:25 <my_server_name> sshd[23396]: Invalid user admin from 123.21.121.162 port 47277
Feb 26 01:07:26 <my_server_name> sshd[24576]: Invalid user setup from 125.212.248.37 port 51559
Feb 26 01:11:29 <my_server_name> sshd[24595]: Invalid user test7 from 125.212.248.37 port 43459
Feb 26 01:15:54 <my_server_name> sshd[24613]: Invalid user squid from 125.212.248.37 port 35362
Feb 26 01:20:36 <my_server_name> sshd[24635]: Invalid user ubnt from 125.212.248.37 port 55512
Feb 26 01:25:30 <my_server_name> sshd[24655]: Invalid user cron from 125.212.248.37 port 47436
Feb 26 03:27:08 <my_server_name> sshd[25046]: Invalid user  0101 from 5.101.40.10 port 40939
Feb 26 03:27:19 <my_server_name> sshd[25050]: Invalid user 0 from 5.101.40.10 port 46872
Feb 26 03:27:32 <my_server_name> sshd[25053]: Invalid user 1234 from 5.101.40.10 port 56612
Feb 26 03:27:40 <my_server_name> sshd[25056]: Invalid user admin from 5.101.40.10 port 42483
Feb 26 03:27:42 <my_server_name> sshd[25059]: Invalid user admin from 5.101.40.10 port 38818
Feb 26 07:42:23 <my_server_name> sshd[26082]: Invalid user admin from 82.209.209.32 port 56889
Feb 26 07:42:27 <my_server_name> sshd[26086]: Invalid user admin from 186.101.223.181 port 58521
Feb 26 07:42:35 <my_server_name> sshd[26088]: Invalid user admin from 109.86.89.70 port 48484
Feb 26 09:20:40 <my_server_name> sshd[26459]: Invalid user ubuntu from 62.210.103.20 port 17592
Feb 26 09:53:32 <my_server_name> sshd[26570]: Invalid user ubuntu from 62.210.103.20 port 33833
Feb 26 10:27:02 <my_server_name> sshd[26696]: Invalid user ubuntu from 62.210.103.20 port 50168
Feb 26 12:22:59 <my_server_name> sshd[27097]: Invalid user alice from 54.197.138.157 port 44960
Feb 26 12:23:05 <my_server_name> sshd[27100]: Invalid user packer from 54.197.138.157 port 46468
Feb 26 12:23:12 <my_server_name> sshd[27103]: Invalid user ec2-user from 54.197.138.157 port 48060
Feb 26 12:23:32 <my_server_name> sshd[27111]: Invalid user deploy from 54.197.138.157 port 52844
Feb 26 12:23:38 <my_server_name> sshd[27114]: Invalid user vagrant from 54.197.138.157 port 54437
Feb 26 12:23:45 <my_server_name> sshd[27117]: Invalid user postgres from 54.197.138.157 port 56043
Feb 26 12:23:58 <my_server_name> sshd[27122]: Invalid user tigertooth from 54.197.138.157 port 59214
Feb 26 12:24:05 <my_server_name> sshd[27125]: Invalid user ubuntu from 54.197.138.157 port 60813
Feb 26 12:24:12 <my_server_name> sshd[27128]: Invalid user centos from 54.197.138.157 port 34175
Feb 26 14:06:56 <my_server_name> sshd[27484]: Invalid user  0101 from 5.101.40.10 port 35918
Feb 26 14:07:04 <my_server_name> sshd[27488]: Invalid user 0 from 5.101.40.10 port 42591
Feb 26 14:07:09 <my_server_name> sshd[27491]: Invalid user 1234 from 5.101.40.10 port 53641
Feb 26 14:07:10 <my_server_name> sshd[27494]: Invalid user admin from 5.101.40.10 port 51921
Feb 26 14:07:19 <my_server_name> sshd[27497]: Invalid user admin from 5.101.40.10 port 41209
Feb 26 14:07:28 <my_server_name> sshd[27499]: Invalid user admin from 5.101.40.10 port 45466
Feb 26 14:07:34 <my_server_name> sshd[27501]: Invalid user admin from 5.101.40.10 port 56275
Feb 26 14:07:39 <my_server_name> sshd[27504]: Invalid user default from 5.101.40.10 port 40792
Feb 26 14:07:47 <my_server_name> sshd[27507]: Invalid user ftp from 5.101.40.10 port 55119
Feb 26 17:40:53 <my_server_name> sshd[28259]: Invalid user pi from 113.232.204.10 port 42721
Feb 26 17:40:53 <my_server_name> sshd[28258]: Invalid user pi from 113.232.204.10 port 42720
Feb 26 19:21:41 <my_server_name> sshd[28896]: Invalid user pi from 115.231.212.82 port 2022
Feb 26 19:21:43 <my_server_name> sshd[28899]: Invalid user PlcmSpIp from 115.231.212.82 port 2408
Feb 26 19:21:44 <my_server_name> sshd[28903]: Invalid user admin from 115.231.212.82 port 2774
Feb 26 19:21:46 <my_server_name> sshd[28907]: Invalid user ftpuser from 115.231.212.82 port 3079
Feb 26 19:21:48 <my_server_name> sshd[28910]: Invalid user ftpuser from 115.231.212.82 port 3456
Feb 26 19:21:50 <my_server_name> sshd[28912]: Invalid user guest from 115.231.212.82 port 3889
Feb 26 19:21:52 <my_server_name> sshd[28915]: Invalid user guest from 115.231.212.82 port 4197
Feb 26 19:21:53 <my_server_name> sshd[28917]: Invalid user guest from 115.231.212.82 port 4645
Feb 26 19:21:55 <my_server_name> sshd[28919]: Invalid user ubnt from 115.231.212.82 port 4966
Feb 26 19:21:57 <my_server_name> sshd[28924]: Invalid user test from 115.231.212.82 port 1611
Feb 26 19:21:59 <my_server_name> sshd[28927]: Invalid user test1 from 115.231.212.82 port 1866
Feb 26 19:22:00 <my_server_name> sshd[28930]: Invalid user test from 115.231.212.82 port 2191
Feb 26 19:22:02 <my_server_name> sshd[28932]: Invalid user test from 115.231.212.82 port 2475
Feb 26 19:22:04 <my_server_name> sshd[28934]: Invalid user test from 115.231.212.82 port 2758
Feb 26 19:22:06 <my_server_name> sshd[28936]: Invalid user admin from 115.231.212.82 port 2999
Feb 26 19:22:18 <my_server_name> sshd[28956]: Invalid user ftp from 115.231.212.82 port 1157
Feb 26 19:22:20 <my_server_name> sshd[28959]: Invalid user ftp from 115.231.212.82 port 1375
Feb 26 19:22:22 <my_server_name> sshd[28961]: Invalid user ftp from 115.231.212.82 port 1569
Feb 26 19:22:24 <my_server_name> sshd[28963]: Invalid user vyatta from 115.231.212.82 port 1933
Feb 26 19:22:25 <my_server_name> sshd[28966]: Invalid user user from 115.231.212.82 port 2156
Feb 26 19:22:27 <my_server_name> sshd[28970]: Invalid user user from 115.231.212.82 port 2446
Feb 26 19:22:29 <my_server_name> sshd[28973]: Invalid user www from 115.231.212.82 port 2952
Feb 26 19:22:31 <my_server_name> sshd[28976]: Invalid user info from 115.231.212.82 port 3242
Feb 26 19:22:32 <my_server_name> sshd[28979]: Invalid user admin from 115.231.212.82 port 3658
Feb 26 19:22:34 <my_server_name> sshd[28982]: Invalid user admin from 115.231.212.82 port 3933
Feb 26 19:22:36 <my_server_name> sshd[28984]: Invalid user git from 115.231.212.82 port 4227
Feb 26 19:22:38 <my_server_name> sshd[28989]: Invalid user vyatta from 115.231.212.82 port 4592
Feb 26 19:22:39 <my_server_name> sshd[28991]: Invalid user operator from 115.231.212.82 port 4880
Feb 26 19:22:41 <my_server_name> sshd[28994]: Invalid user webmaster from 115.231.212.82 port 1144
Feb 26 19:22:43 <my_server_name> sshd[28997]: Invalid user nagios from 115.231.212.82 port 1472
Feb 26 19:22:45 <my_server_name> sshd[29000]: Invalid user oracle from 115.231.212.82 port 1937
Feb 26 19:22:46 <my_server_name> sshd[29003]: Invalid user fax from 115.231.212.82 port 2250
Feb 26 19:22:48 <my_server_name> sshd[29008]: Invalid user fax from 115.231.212.82 port 2530
Feb 26 19:22:50 <my_server_name> sshd[29010]: Invalid user sales from 115.231.212.82 port 2753
Feb 26 19:22:52 <my_server_name> sshd[29013]: Invalid user server from 115.231.212.82 port 3009
Feb 26 19:22:54 <my_server_name> sshd[29016]: Invalid user mysql from 115.231.212.82 port 3348
Feb 26 19:22:55 <my_server_name> sshd[29019]: Invalid user public from 115.231.212.82 port 3621
Feb 26 19:22:57 <my_server_name> sshd[29022]: Invalid user demo from 115.231.212.82 port 3875

This is the output of netstat -l

Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State      
tcp        0      0 0.0.0.0:ssh             0.0.0.0:*               LISTEN     
tcp6       0      0 [::]:ssh                [::]:*                  LISTEN     
udp        0      0 0.0.0.0:bootpc          0.0.0.0:*                          
Active UNIX domain sockets (only servers)
Proto RefCnt Flags       Type       State         I-Node   Path
unix  2      [ ACC ]     STREAM     LISTENING     10332    /run/systemd/private
unix  2      [ ACC ]     SEQPACKET  LISTENING     10343    /run/udev/control
unix  2      [ ACC ]     STREAM     LISTENING     10345    /run/systemd/fsck.progress
unix  2      [ ACC ]     STREAM     LISTENING     12394    /var/run/nscd/socket
unix  2      [ ACC ]     STREAM     LISTENING     12396    /var/run/.nscd_socket
unix  2      [ ACC ]     STREAM     LISTENING     10351    /run/systemd/journal/stdout
unix  2      [ ACC ]     STREAM     LISTENING     12176    /var/run/dbus/system_bus_socket
unix  2      [ ACC ]     STREAM     LISTENING     28848    /run/user/1000/systemd/private
unix  2      [ ACC ]     STREAM     LISTENING     28853    /run/user/1000/gnupg/S.gpg-agent
unix  2      [ ACC ]     STREAM     LISTENING     28856    /run/user/1000/gnupg/S.gpg-agent.browser
unix  2      [ ACC ]     STREAM     LISTENING     28858    /run/user/1000/gnupg/S.gpg-agent.ssh
unix  2      [ ACC ]     STREAM     LISTENING     28860    /run/user/1000/gnupg/S.gpg-agent.extra
1
  • "network in-traffic has jumped by 10x and the server stopped to accomplish..." - The failed login attempts are just the reality of having an open ssh port on the internet. This wouldn't cause traffic to spike like that or cpu load to increase enough to matter. Look elsewhere for the root cause of your problem. Unless you have a weak password and someone actually managed to get in, in that case you need to nuke it from orbit. Feb 27 '18 at 20:59
1

As noted, if you expose SSH, you're going to get this. In addition to the basics (disable SSH root login, only allow public key auth/disable password auth), disable root login, only allow SSH v2, trim allowed ciphers to known good ones[0]), you can:

  1. Put SSH on a non-standard port (2201 or something) so that the simple automated scanners will miss it. This will at least cut the volume.
  2. Run something like fail2ban to block persistent attempts. Not as useful as it once was, as more and more scanners are randomizing src IPs.
  3. Use IPTables or ufw to block SSH attempts from anything other than authorized networks. Works great if you know beforehand what networks you're going to be coming from, even if you have to allow a whole /12 or something to cover your whole cable modem/DSL block.
  4. Use geolocation with GeoIP and IPtables or fail2ban to block attempts from countries that have no business trying to log in over ssh. In my experience, this usually significantly reduces but does not eliminate volume. YMMV.
  5. Use IPTables to rate limit SSH attempts from any given IP. Not as useful as it once was, as more and more scanners are randomizing src IPs.

N.B. - sshd_config has a setting called MaxStartups that looks like it's a rate-limiting mechanism. It kinda is, but it has global scope, so if the script kiddie is hammering you, you are going to get DoS'ed.

[0] There are many sshd best practices checklists, but I've seen very few that address the issue of cipher suites like https://stribika.github.io/2015/01/04/secure-secure-shell.html

[1 ] See: https://manpages.debian.org/stretch/openssh-server/sshd_config.5.en.html#Port

[2 ] See: https://www.fail2ban.org/wiki/index.php/Main_Page

[3] Something like iptables -I INPUT -p tcp ! -s yourIPaddress --dport 22 -j DROP

[4] https://www.axllent.org/docs/view/ssh-geoip/

[5] Something like:

iptables -A INPUT -p tcp --dport 22 -m state --state NEW -m recent --set --name SSH --rsource

iptables -A INPUT -m recent --update --seconds 600 --hitcount 3 --rttl --name SSH --rsource -j DROP

to drop more than 3 tries in 10 minutes. You could also use something like hashlimit to limit the rate of packets from a particular source rather than connections.

0

Many attackers scan ip addresses and try to log in if they find anything. By virtue of having an externally accessable IP address, your server is being brute forced by (likely) automated attacks. I recommend making sure that you have a strong login password or (even better) make ssh ligin only possible via a ssh keypair

1
  • Yes, only ssh login is available.
    – Julpi
    Feb 27 '18 at 19:01
0

When attacking a server the first step is recon, and that involves port scanning. Most script kiddies will run a port scan that just blasts the network with requests trying to see what ports are open. They are scanning pretty high ports so it looks like they are looking for some open webserver specific applications or ports. Make sure SSH, RDP and Telnet are not open, if SSH is then make sure it isn't a default credential. You can allways block their IP addresses from making requests.

0

This isn't a honest mistake. If it was, whoever is on the other end would recognize their mistake pretty soon when they can't log in, or when whatever services need something like a SSH tunnel stop working. You also wouldn't be seeing a gazillion different usernames tried.

It also isn't likely to be targetted.

Rather, as unfortunate as it is, it's a fact of life on the Internet these days that by allowing traffic in, you're going to get unwanted traffic. Sorry, but the best we can do is to try to deal with it in ways that minimize the inconvenience to legitimate users, while keeping out as much of the unwanted traffic as possible as close to the network as possible (to reduce both log clutter and attack surface).

Setting up your SSH server to enforce public-key authentication can be good advice in general, but won't help much in this case, because the attacker isn't even hitting valid usernames, let alone passwords. So while enforcing public-key authentication can be a good thing to do for other reasons, it won't help you with this problem.

Instead, I'd suggest three things.

  • Install fail2ban. It's in the Debian repositories, is pretty much install and forget, and it will automatically block traffic coming from specific IP addresses for a period of time if they hit you with, in this case, too many unsuccessful logins. It's also easy to adjust to your particular situation if the stock rules aren't a good enough fit.
  • Restrict the set of users that can log in via SSH. For OpenSSH, you can use AllowGroups and AllowUsers in /etc/ssh/sshd_config for this, possibly in combination with a Match block. This way, even if an attacker hits a username that is valid on your system, they need to hit one that is allowed to log in via SSH. Consider setting up a separate group for those users allowed to log in via SSH, then adding the required users to that group and adding AllowGroups groupname to sshd_config. At the very least you should set up your SSH server to refuse logging in directly as root.
  • Run SSH on a non-standard port. This isn't a panacea and it isn't a security measure by itself, but it will give you some respite against many automated scans. Make sure to adjust any firewall rules, including the configuration for fail2ban, accordingly!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.