We have many Linux servers in several co-locations over the world. By design, these machines generally should not be able to access the Internet. Depending on location and role, some sites may have very limited Internet access (e.g. default deny with one or two sites white listed). So depending on the circumstances, the no-Internet policy is configured either by routing rules at the switch level, or by a filter device (e.g. Untangle).
One team is responsible for configuring the network per policy, and a separate team (me) is responsible for periodic auditing to make sure things are actually working as intended.
The question, therefore, is how to comprehensively check that the Internet cannot be reached? The naive method is to simply "ping google.com". But what if Internet is available, but DNS is simply not working? OK, so ping a known public IP, e.g. "ping 8.8.4.4". But what if ICMP is filtered, but e.g. TCP connections are still allowed? What if TCP and UDP are filtered, but raw IP is not? What if the network team left themselves a little hole (not necessarily by malicious intent, maybe honest mistake, maybe bug in firewall or switch)?
Obviously, ping is a nice easy check when you want Internet access, but it's woefully insufficient for ensuring that Internet is completely blocked.
The ideal approach would be to try every possible combination of
- Port
- Public IP
- Protocol
But that's not practical. But I'm looking for a reasonable compromise between that and a naive ping test.
My thought was to try to use nmap to look for any open ports on public IPs that should generally be up---public DNS servers. So, for example (8.8.4.4 is a public DNS server):
# nmap -v -v -v -T3 -sS 8.8.4.4
Starting Nmap 6.40 ( http://nmap.org ) at 2014-03-18 10:24 CDT
Initiating Ping Scan at 10:24
Scanning 8.8.4.4 [4 ports]
Completed Ping Scan at 10:24, 0.01s elapsed (1 total hosts)
Initiating Parallel DNS resolution of 1 host. at 10:24
Completed Parallel DNS resolution of 1 host. at 10:24, 13.00s elapsed
DNS resolution of 1 IPs took 13.00s. Mode: Async [#: 2, OK: 0, NX: 0, DR: 1, SF: 0, TR: 4, CN: 0]
Initiating SYN Stealth Scan at 10:24
Scanning 8.8.4.4 [1000 ports]
Completed SYN Stealth Scan at 10:24, 0.07s elapsed (1000 total ports)
Nmap scan report for 8.8.4.4
Host is up (0.0015s latency).
All 1000 scanned ports on 8.8.4.4 are closed
Read data files from: /usr/local/encap/nmap-6.40/bin/../share/nmap
Nmap done: 1 IP address (1 host up) scanned in 13.14 seconds
Raw packets sent: 1004 (44.152KB) | Rcvd: 1001 (40.040KB)
This was run from a server where the Internet access block is believed to be working correctly. Yet why does it say "Host is up"? How does it know? A simple ping fails, and I am unable to use that server with dig.
I was hoping to get away with a simple script that looked for "Host is up" to flag Internet access, but it looks like that would generate too many false positives.
Anyone have any better suggestions or approaches to this problem?