3

I have a Centos 7 server running httpd for a webmail service. I'm blocking outgoing connections, including ports 80 and 443. I'm doing this with iptables on the machine.

Now looking at the logs I see the server is trying to make outgoing connections to ports 80 and 443.

Using --log-uid in iptables I was able to see that the user making those connections is apache which is the user running httpd.

The connections occur at random times throughout the day and to about 50 different IP addresses on port 80 and more than double that on port 443.

How can I determine what process is making those connections?

And is there any way to determine what's going on without actually allowing those connections and capturing the traffic?

  • Have you done a whois on the IP's? Its not uncommon for servers to reach out to update servers to see if there are software updates. – DKNUCKLES Jan 10 '18 at 14:44
  • Yes, but that didn't reveal much. Some large ISP owned blocks, some Google IPs and some random IPs. I have now captured some of that traffic (by temporarily letting it go out) and so far they are all GET requests for either fonts or css files. The fonts are google fonts, but the css is "random", i.e. not some general css from a CDN. What I'd really like to hear is more the general approach to solving a problem like this rather than this specific problem. – IamNaN Jan 10 '18 at 15:59
3

In addition to tracking the process down with netstat or lsof, on the affected system, instead of blocking the traffic, you could use iptables to redirect the traffic to services running on ports 80 and 443 that you control. This way, you could potentially gain some insight into HTTP headers and what kind of content is being accessed.

1

Yes there are ways to do it.

  • tcpdump the traffic to see what kinds of packets are being issued, from what port.
  • lsof - the processes to see which one is using that port.

Combine running lsof periodically (every 10s or every 1s) with crontab and logging the output to file would also give you some understanding on when those requests occur.

A misconception in your last statement is that you need to allow the traffic to capture it. Just capture it on the local machine. This will give you ideas what to investigate further.

I would guess that you might have some apache virtual servers issuing some healthchecks that are not disabled. Or a human connecting to the apache user and issuing curl requests. But that is just a guess, always verify.

  • Thanks for the reply. If the traffic is blocked all I'm going to see on a capture is the SYN packets going out. How would I use lsof? Wouldn't I have to run it while such a connection attempt is occurring? I suspect the requests are from httpd for remote resources used by the app but I don't know for sure and am generally interested in finding out how to track this down, not just for this specific scenario. – IamNaN Jan 10 '18 at 12:26
  • Yes, you would see SYN packets only. This would allow you to know the destination IP, then you simply open that IP:80 or IP:443 to see what that process is trying to access. If you know it already then just do that. For lsof -> sudo lsof -i -P -n would show the processes occupied ports, maybe apache is listening on more ports than you know? Anyway, durring the time the request to port 80 or 443 is initiated you should see some ephemeral port being opened and the connection in SYN-SENT state. – Mindaugas Bernatavičius Jan 10 '18 at 12:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.