What is this weird localhost IP traffic on multiple ports in the 5400-5600 range?

Question

I am encountering an issue I recently have seen in my Fedora 23 machine (latest updates). I am getting about 130 IP packets passed back and forth over the lo interface in one go, every 1 second (like a timed process). It occurs for about 2-5 minutes, then stops for a few seconds to a couple mins., and then starts right back up again. I can see the packet traffic in a tcpdump capture on the lo interface, but I can't find any real information on the particular ports it is using other than IANA names, and I am not an expert on protocol debugging with tcpdump. A couple sample messages are pasted here, but I will also include a pointer to a paste bin for a more complete capture (yes, I read the SE general opinion on *bin sites, but it is a huge capture even for a couple seconds, and enough should be here for future searches to find):

21:33:43.047410 IP localhost.localdomain.5665 > localhost.localdomain.44574: Flags [R.], seq 0, ack 2284755030, win 0, length 0
21:33:43.047448 IP localhost.localdomain.38818 > localhost.localdomain.5667: Flags [S], seq 1778200592, win 43690, options [mss 65495,sackOK,TS val 9036387 ecr 0,nop,wscale 7], length 0
21:33:43.047460 IP localhost.localdomain.5667 > localhost.localdomain.38818: Flags [R.], seq 0, ack 1778200593, win 0, length 0
21:33:43.047496 IP localhost.localdomain.37068 > localhost.localdomain.5669: Flags [S], seq 3846609184, win 43690, options [mss 65495,sackOK,TS val 9036387 ecr 0,nop,wscale 7], length 0
21:33:43.047508 IP localhost.localdomain.5669 > localhost.localdomain.37068: Flags [R.], seq 0, ack 3846609185, win 0, length 0
21:33:43.047544 IP localhost.localdomain.44684 > localhost.localdomain.amqps: Flags [S], seq 2934840620, win 43690, options [mss 65495,sackOK,TS val 9036387 ecr 0,nop,wscale 7], length 0
21:33:43.047556 IP localhost.localdomain.amqps > localhost.localdomain.44684: Flags [R.], seq 0, ack 2934840621, win 0, length 0
21:33:43.047592 IP localhost.localdomain.56366 > localhost.localdomain.jms: Flags [S], seq 3939722005, win 43690, options [mss 65495,sackOK,TS val 9036387 ecr 0,nop,wscale 7], length 0
21:33:43.047604 IP localhost.localdomain.jms > localhost.localdomain.56366: Flags [R.], seq 0, ack 3939722006, win 0, length 0
21:33:43.047640 IP localhost.localdomain.50540 > localhost.localdomain.v5ua: Flags [S], seq 3048240647, win 43690, options [mss 65495,sackOK,TS val 9036387 ecr 0,nop,wscale 7], length 0
21:33:43.047652 IP localhost.localdomain.v5ua > localhost.localdomain.50540: Flags [R.], seq 0, ack 3048240648, win 0, length 0
21:33:43.047688 IP localhost.localdomain.57428 > localhost.localdomain.questdb2-lnchr: Flags [S], seq 2388676920, win 43690, options [mss 65495,sackOK,TS val 9036387 ecr 0,nop,wscale 7], length 0

It seems to go in sequence, like a port scan, but it always stays within the 5400-5699 port range. More is available at the Fedora Project paste site: https://da.gd/Zr1b . I could only paste up to 20 seconds of capture because there is so much traffic!

I normally would look up the process in a lsof and track down the culprit, but the ports go by so fast that lsof misses whatever it is that opens the connection. Netstat is no help either, or at least I may be using it wrong. Some other troubleshooting steps I did take was to shut down networking, and pulled the Ethernet cable. No dice. I also ran up an instance of ntop, directing it to attach to the lo interface, but it could not see a thing.

So, does anyone have a clue as to what this is? I can say I noticed it in my network widget on the desktop when I got home from work today, and I had updated the machine the day before - but there was no localhost traffic I saw then.

Answer 1

I got the time to track it down, and it looks like I found the answer. It's 'adb', the Android Debug Bridge service, installed with the Google Android tools package. I had put it on my machine to help unbrick my daughter's phone (don't ask) and forgot to shut it off. But that was months ago, and like I said I didn't see the traffic in my desktop widget (Gnome/MATE System Monitor applet) until after I updated the OS on Apr. 19th, and not immediately after the update. I am digging through the release notes on the list of updated packages now to see if there was a networking-or-other package that updated something to bring this up on the radar.

Anyhow I'm recording the steps I took here in case someone wants to know how I found it, maybe it'll help someone someday. I started by using Wireshark, but not really knowing what to look for all I could note was that the pattern started at port 5401, ran every odd numbered port up to 5699, and then recycled. Then I went back to the basics and looked up the man page for lsof. I ended up using the lsof command like so:

lsof -i :5000-5700

It shows all processes using internet ports 5000 through 5700. A couple processes showed up, 'adb' being one of them (listening on port 5037) . 'avahi-deamon' showed up too, running UDP on mDNS (port 5353). I don't run Zeroconf/Bonjour/whateverthenameisnow, so I shut down avahi-daemon. No dice, still had the traffic. Then I went to look up adb, it had no man page, no apropos, no whatis, and a 'dnf whatprovides /bin/adb' came up empty. Strange. However running a 'strings' on it and piping it to less showed the word ANDROID show up a few times. Bingo. I took a look at the services list in systemctl, and saw 'adb.service' was enabled. I shut it off with 'systemctl disable adb.service', and wala, like magic the traffic stopped.

Now, doing that was just a shot in the dark, I did not know for sure it was adb that was making the traffic before I shut it down. After all I could only prove it was listening on port 5037. I wonder what it is doing looking through all those ports. Obviously trying to connect to something. When I get more time I will look through the documentation.

Answer 2

It looks like you're seeing a LOT of bad connections (syn connection immediately followed by rst flags). This means you're trying to make a connection to yourself via localhost, and the connection is being closed (reset) by the TCP/IP stack.

By any chance are you using a hosts file to redirect ad/malware domains to localhost (172.0.0.1)? That would cause this behavior, as the domain resolves to the localhost, then tries to make a connection, but that port doens't have an active process watching it, so the connection is reset by the kernel (contains the implementation of your tcp/ip stack)