How can I portscan large subnets?

Question

I need to scan several /16 and a /8 subnet to see what hosts are up and additionally, if any services are running on a select few ports. I've tried various tools such as angry IP scanner, Nmap, advanced IP scanner. But they all crash out after a while and none of them complete the scans as required.

Whats the best way to identify a servers and services in a large subnet?

Answer 1

Unicornscan is a tool known for high-speed scanning of large net blocks (e.g. last I observed was sustained 60+mbps internet-based scans).

Nmap in the past definitely had memory consumption issues w/large scans, but I haven't seen those issues in the 5.x/6.x versions when performing 65k ports scans over several /16s.

The memory problems I run into are typically due to ip_conntrack and below are a couple of recommended links to tweak kernel/ip_conntrack settings that can help nmap:

• ip_conntrack: maximum limit of XXX entries exceeded (I have experimented with values of 256k+ w/apparent success)
  (also check out Tuning Linux firewall connection tracker ip_conntrack)
• Disabling ip_conntrack to save memory, restore happiness

Answer 2

Well as @TerryChia says I'm surprised that nmap is causing problems for you, I don't think I've ever seen it crash..

However that said, assuming it is, there's a couple of approaches you can take which may help.

1. Manually break up the range. Create a text file with smaller ranges and then automate the process of kicking a scan of each range off one at a time. Once you've got all the results you can aggregate them together to give an overall picture. An example of a script you can use to parse a directory full of nmap results is here
2. Having not seen the switches you're using it's hard to provide exact advice here, but if you're not already I'd be using -sP and a range of ports (specified via -PS or -PU) to establish a list of "up hosts" which you can then scan, if you need more information.

Answer 3

Have you investigated the reason for the tools you use crashing after a while? I have used nmap to scan rather large networks without any issues whatsoever.

In fact, the nmap tool has been used to port scan large portions of the internet without any problems as seen in this defcon presentation.

You might want to experiment with different nmap flags or try scanning with a different machine to test out the different possibilities why your scans might be failing.

If you really cannot scan the entire subnet at one go, split the task into several scans. It should be easy enough to do...

Answer 4

I believe nmap is suitable for scanning large number of IPs. I myself have scanned only /16 subnets with nmap but HD Moore in 2012 scanned the entire Internet through nmap. All you need is play with the command line arguments of nmap. The ones HD used for scanning the Internet were:

1. --min-rate=5000 -m 256 --min-host-group=50000 -PS -p
2. Match --min-rtt-timeout to --max-rtt-timeout

So I guess nmap can be quite effective. If it is capable of scanning the entire IP range over Internet, it won't have any problem fulfilling your requirements.