Lots of books talk about using tools such as whois and other information gathering tools to collect information on the network and of course, not forgetting nmap. However, in a real network with such a large number of hosts, wouldn't there be an information overload? How do pentesters know which host to start probing in the sea of machines? What would a good strategy be in terms of filtering the information and focusing only on whats important?
All security tests will have a scope and its this scope that would typically drive the process of the review.
So I'm assuming from the question that you're thinking about a network based review as opposed to a review of a specific application. With this kind of review the starting point is likely to be a set of IP address ranges that are in-scope of the test. So from that starting point, one approach would be
a) Do a light port scan for common ports (something like the nmap top 1000) across your range
b) Once that's complete kick off more comprehensive scans and at the same time start reviewing the open ports from the initial scan.
c) Do some automated vulnerabilty scanning on the open ports to find low-hanging fruit.
d) Start working through the open ports doing manual review of each one. How far you take this will very much depend on the time available for the engagement and the number of open ports found. If you've only got a small number you can afford to go quite deep in trying to attack each port, however if you've got 1000's of open ports and a limited time window, you're probably going to want to automate as much as possible to get coverage.
e) once your comprehensive port scans have completed, review these for less common ports, which could be useful targets.
This is the kind of flow you might expect on a "standard" pen. test, obviously if you're doing something more like a red-team where you want to avoid detection, there would be a lot less automation going on as automated scanning tools tend to be very noisy.
The question is selective & is about approaches to the penetration testing methodology followed on a large scale network. To be precise, there needs to be a scope of work which could be mapped something like below:
This will be first hand information gathering phase & if you're into an Enterprise Network, I assume - all such info could be retained by the project manager or the SPOC who shall be handling the Penetration Test. Make sure to understand if the pentest or vulnerability assessment is
grey-box. If it's glass-box - the project managers & few select team are meant to witness the pentest & wherever required send inputs to the red teamers.
I want you to deal your problems with information. The more information you collect, collate & regulate - the more in-depth coverage approach with clean slate your team will be able to succeed in a penetration test coverage. For an instance, after having chalked out
ranges of IP's for selective penetration tests, one should be able to trigger
netdiscover from the specific segment which's in the scope & chalk out few front applications as well as shown below in images (to be clear to the point!!):
This could next lead to security testing the broader perimeters of all interfaces one by one with a greater insight & success hits. Focusing on only what's important is only clear when you actually do hit those interfaces while assuming it's a
black-box. If its not - priorities could be set by your immediate manager on client's end or premises.