I had read through several research papers that discussed the trend of existing automation scanners to produce false positive results. I also read through some articles that showing the manual penetration testing could be solution to false positive results by manually identifying each investigated exploits.

So, here is my question. Are the false positive results produced by automation scanner a problem in penetration testing? If the answer is 'yes', from which perspective it will affect the penetration tester job in conducting the testing?

  • Can you provide some reference ... – Ali Ahmad May 2 '13 at 12:08
  • Reduced down to the core of your question. If you have follow up questions please ask them separately. – Rory Alsop May 2 '13 at 13:36

You ask a lot of different questions.

I would say that as a whole, an automated tool that produces 100% accurate results is impossible. Else all the penetration testers in the world would be out of a job.

To give one example, RedHat backports security patches to older versions of software under their long-term support. An automated tool might detect that the server is running a particular kernel version that has a privilege escalation exploit and report it as such. However, due to RedHat's security patches, that particular privilege escalation vulnerability is fixed and no longer exploitable. A penetration tester however, would use that automated tool's report as a starting point in the exploitation process and find that the vulnerability is actually fixed. This tends to remove a lot of the false positives produced by automated tools.

Sure, weeding out false positives does consume a fair amount of time in a penetration test. However, I wouldn't consider it a downside. It is just part of the whole penetration testing process. It probably will not cause any delays in the penetration testing process or affect it's quality as a good penetration tester should have already taken into account these factors at the start of the test.

A good penetration tester uses the tools available to him to make his job more efficient. Automated tools are no replacement for proper skills, knowledge and experience.


Scanners are perhaps more limited in the areas that they can cover than false positives that they produce.

A false positive while testing can be annoying and time consuming, but to a large extent an experienced tester can compensate easily as you become more familiar with the likely false positives (e.g. the example that @Terry gives of backported security fixes) which makes the less of a problem.

In terms of automated scanners there is a tuning balance to be made by the authors between false positive rate and false negative rate. Where a scanner sees something which "could" be an issue but which can't be definitvely confirmed, they can flag it (higher false positive rate), ignore it (higher false negative rate) or do what I think a lot of scanners do (especially web app scanners) and assign a level of confidence to the finding.

This again feeds into the manual process as a tester can take high confidence findings as not requiring much follow-up and focus on findings which may be high-impact but which the scanner is unsure about (best use of manual efforts).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.