2

One of our client's routers is receiving packets from the same source IP address and port repeatedly, even when the router is rebooted and obtains a fresh dynamically assigned IP address, and while no other traffic or connections have yet been made. The router is configured to silently drop all unauthorized inbound packets and does not respond to unauthorized inbound packets (black hole, and no half-acks).

The IP in question is 45.227.254.22, registered in Belize. The source port is always the same (TCP 52383), yet the destination port it attempts to connect with can vary, and can start at TCP 20000 and go up from there.

Out of the thousands of random pings, scans, and pokes the router receives on a daily basis from across the internet, this is the only address that repeatedly and persistently appears in logs, and continues attempts to connect even when the client's router itself is rebooted and has a freshly assigned previously unknown IP from the ISP, and no other outbound or inbound connections have yet been made.

It would appear that what ever is at that IP address is following the router.

Sample From Log:

The client's IP has been anonymized below as 00.00.00.00 to protect privacy. The client router is located in North America (YYZ).

20:45:31.495556 IP 45.227.254.22:52383 > 00.00.00.00:48801: Flags [S], seq 1102121663, win 1024, length 0

20:45:42.621963 IP 45.227.254.22:52383 > 00.00.00.00:31373: Flags [S], seq 3756604334, win 1024, length 0

Questions:

  1. What could be causing this?

  2. How does it always find the client's router?

FAQ:

(This FAQ will be updated as needed)

Q. Who is managing and operating the router, the client or the ISP?

A. The router is fully managed and operated by the client.

NOTE: The answer provided below by @samy-kamkar is not considered the correct answer as typical scanning bot behaviors are well known and were ruled out early on. The IP was studied on the AbuseIP database and other places prior to this posting as well. It is certainly a suspicious IP. However, it having been reported already by others does not conclude that it is only a simple scanning bot.

The suspicious IP is noted to have a range of malicious behaviors, including scanning, brute forcing, and outright hacking attempts.

The client's router is hammered with scans, pings, prods, pokes, and even martian packets, 24 hours a day, 7 days a week, and that is considered normal for anything connected to the public internet. This particular situation does not point to any normal bot. Out of the many thousands of unknown inbound connection attempts, this is the only persistent suspicious IP address out of all others that rapidly and repeatedly appears across a large range of dynamically assigned IP addresses.

The client IP potentials it is targeting with our client are massively large, in the order of tens of thousands of possibilities because of the diverse network blocks assigned to customers by the ISP.

As an example, one connection could be assigned from a range of 10.11.0.0-10.11.255.255, another re-connect could assign from a range of 20.30.0.0-20.30.255.255, and yet another re-connect could assign from a range of 40.50.0.0-40.50.255.255, and so on, and there are dozens of such possible ranges. The suspicious connection also attempts a range of destination ports, but always from the same source port. This behavior is also different from other scans hammering the client's router.

When the client router does connect to the internet with a fresh dynamically assigned address from any random vast range, the suspicious IP begins it's attempts within 30 seconds or less. That is not normal.

Improve this question
18
  • The client's router maybe can be found by the router fingerprint, scanning the whole network/range of IP assigned to that ISP? Is it listed on Shodan.io? Or maybe it's a rogue employee?
    – Azteca
    Mar 22, 2019 at 22:04
  • Please elaborate on the router fingerprint. Shodan wouldn't show the client router cause it gets a fresh IP each reboot. Rogue employee is not a factor here.
    – RealDrGordonFreeman
    Mar 22, 2019 at 22:08
  • Well... a ping sweep on the hole network could retrieve the TCP/IP fingerprint or even with ssh, the ssh-rsa fingerprint, search for Banner grabbing or TCP/IP stack fingerprinting a simple nmap -O -v <target_ip> would do the trick I think.
    – Azteca
    Mar 22, 2019 at 22:20
  • Thanks, I will research this. SSH is disabled by the way. That leaves TCP/IP stack, but ICMP and pings are blocked as well. Also note that the client router's IP addresses can be assigned from a very large and random pool by the ISP. For example, one reboot would give 10.11.12.13, and the next reboot would give 55.66.77.88.
    – RealDrGordonFreeman
    Mar 22, 2019 at 22:25
  • Still no conclusive answer. The very first post-boot new connections were also checked, as well as refreshing the client router's IP without a reboot. In all cases, the suspicious IP found the router and attempted contact again. This is baffling.
    – RealDrGordonFreeman
    Mar 23, 2019 at 17:47

4 Answers 4

Reset to default
0
+100

The most likely scenario is it's just a scanner that's scanning your ISP over and over. But this is still just an educated guess.

The only way to be certain is to gather more data. You could create a honeypot where any traffic from the offending remote IP address is directed to an endpoint that responds and completes the TCP connection. Run wireshark on the endpoint and see what, if anything the other end responds with. If it doesn't respond with anything, it's very likely just a scanner. If it responds with more data, you'd have to do further analysis of what it sent.

For your honeypot you'd need to listen on a LOT of ports to get this to work. You could write a simple program that listens on a range of ports, or use a combination of iptables and netcat that uses iptables to redirect a port range to a single port, and then have netcat listening on that port.

This may not be worth your, or your clients time or money to setup. But it's likely the only way to know what the traffic is.

Improve this answer
14
  • Maybe they just could block all outbound traffic over the wan interface in order to do a first troubleshooting. If the incoming traffic from the malicious ip stops after a reboot (and ip renewal), they could do a further analysis with a honeypot.
    – mcruz2401
    Mar 26, 2019 at 14:19
  • @mcruz2401 That might be useful as well. It has the downside of stopping all internet traffic, which means you have to have a maintenance window where you can turn off the internet for a time (which depending on the scenario may or may not be possible).
    – Steve Sether
    Mar 26, 2019 at 14:32
  • @SteveSether So only one issue with your answer, it is not the same subnet. It is across a very massive range of subnets. See my question and comments for how large. It is also the same source port and exact same IP, as if it were a server like a control point. I would ask that you update your answer with concern to your claim that it is scanning the same subnet when it is not doing that.
    – RealDrGordonFreeman
    Mar 26, 2019 at 14:40
  • I understand this, but the troubleshooting is proposed based in fact that they have been rebooted the device several times.
    – mcruz2401
    Mar 26, 2019 at 14:45
  • 1
    @mcruz2401 It's possible the router is infected. The more likely scenario is still a scanner scanning the whole ISP, and that likelyhood went up when the traffic returns when blocking outgoing traffic. As doctors put it, when you hear hoofbeats, think horses, not zebras.
    – Steve Sether
    Mar 27, 2019 at 17:57
6

How much of the public IP is changing upon the DHCP renew? How quickly before you see the packets upon getting a new IP?

This is unlikely a targeted attack and more likely a shotgun-approach scanner bound to scanning a large subnet. It only takes a few minutes to scan an entire class B with SYNs.

The IP in question has been reported by several as an abusive IP already: https://www.abuseipdb.com/check/45.227.254.22

Improve this answer
3
  • 2
    This is by far the most likely scenario. If @RealDrGordonFreeman wants to be more certain, you could setup a honeypot for that incoming IP and forwarding traffic from that IP to some device that completes the TCP connection and see what (if anything) the other end responds with, while capturing the whole conversation with wireshark. If it returns no further packets on the connection, it's just a scanner. This is unlikely to be worth your clients time and money, but it's likely the only way to know exactly what's going on.
    – Steve Sether
    Mar 25, 2019 at 17:22
  • Yes, I did previously see it was reported there. The fact that it is reported, including for Brute Force attempts, only increases the seriousness of this. The client's router, like most routers, is scanned thousands of times per day from many places. That is normal. What stands out about this particular IP is it's persistence across vary large and different dynamic ranges, where no other similar persistence against the client's router is noted from anywhere else.
    – RealDrGordonFreeman
    Mar 25, 2019 at 20:12
  • @SteveSether Can you go ahead with converting your honey pot idea comment to a formal answer, minus your comment that a bot is the likest scenario. No bot can scan this many multiple vast ranges within 30 seconds or less. So your answer could be that because of the vast dynamic and disparate ranges of IP's, and because the suspicious IP shows up within 30 seconds or less, the best way to diagnose this very difficult and unusual problem is to create a honeypot, complete the connection, and study the data. Once you complete your answer, I will mark that as the answer.
    – RealDrGordonFreeman
    Mar 26, 2019 at 12:27
0

If you don't see any other outbound or inbound connection right after a reboot, maybe the router itself has been affected by some kind of apt like be2, doing callbacks after restart. In the other hand, the dump of connections shows something like port scanning behavior: same origin port, different destination ports and only syn packets without ack responses (there's no established connections?) Btw, the destination ports are in the range of private ports. Just in case you have any kind of previous traffic after reboot, that might indicate a callback to a c2 located in the malicious ip, upcoming from any host behind your router. Anyway, the best practice is to add a firewall rule that blocks the traffic from/to the suspicious ip.

Improve this answer
2
  • Thanks for the suggestions. The problem of inbound/outbound rules here is that the router itself is the firewall. If it is infected, any good malware would take care of getting around those rules or acting before any rules are loaded. The good news is it is saying it is silently dropping packets from the suspicious IP, so if it is malware, it might not have found a way around the inbound, perhaps only the outbound to initiate the first call home.
    – RealDrGordonFreeman
    Mar 26, 2019 at 11:34
  • That is right, a good malware use dga's, not hardcoded ip addresses. In order to troubleshoot this issue, you need to add a rule to block all outbound traffic over the wan interface, and then reboot. Make sure that a new ip has been leased by the ISP, and check the logs. If no traffic incoming from the bad ip, the callback theory from router/host is the correct one. Otherwise, the massive syn scan with banner grabbing could be the answer. There is a good practice in networking that consists in block all incoming traffic from wan, except for established and related connections. Good luck!
    – mcruz2401
    Mar 26, 2019 at 13:51
0

The IP is from Panama and has been reported to be abusive: Sql/code injection probe / SQL Injection Attempts / Command Injection.

Just deny it completely by standard ACL and you're fine.

It does find your new IPs because it scans full IP classes and the class of IPs you use from your ISP is included in its scans so no matter what IP your router takes it will still be in a class included in the scans.

Improve this answer
3
  • That would be tens of thousands of addresses within 30 seconds across multiple blocks. Do you have any further information on such a possibility?
    – RealDrGordonFreeman
    Mar 26, 2019 at 11:26
  • It is not unusual; I encounter this often enough. I can scan from a single standard average PC /24 blocks at an average the rate of 1 per second, so that is nothing spacial. Imagine spawning 64k threats and scanning with like that....that would already be 256 /24-blocks / second.
    – Overmind
    Mar 26, 2019 at 13:18
  • You are claiming a definitive conclusion for something which is unknown at this point. Because the IP has been reported previously does not make it any less of a problem. It has been noted doing more than just SQL injection attempts. Also, 30 seconds to connect over tens of thousands of very different ranges would attract the attention of the ISP and others. The other issue being the suspicious IP is using the exact same IP and source port against the client's router each and every time, as if it is a server, such as a control point. Waiting to hear from ISP of what they see over there.
    – RealDrGordonFreeman
    Mar 26, 2019 at 14:21

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .