Is running an automated application vulnerability scanner sufficient to certify our application is secure? Why isn't it?
From the original Area51 proposal.
Information Security Stack Exchange is a question and answer site for information security professionals. It only takes a minute to sign up.Sign up to join this community
Sure automated checks are not sufficient, and believed that never will be. Well, until some AI is made. As for example fuzzers - they can cover only checks that fails for harsh input. Logical bugs will be missed. Source code analysis shows good results when the talk is about really severe bugs and those vulnerabilities that requires several conditions to meet. Binary analysis might be daunting for not so experienced person, but it is possible to inspect closed source code application and besides, it takes into account compiler specifications that are impossible to diagnose during source code analysis. So, every method has it advantage and should not be used alone.
Relying only on automated vulnerability scanning to vet the security of most applications is woefully inadequate.
Why? Because automated tools will miss vulnerabilities (type II errors/false negatives).
The uglier implication of this question is the application’s security is important yet it was ignored during development.
Applications cannot be certified as secure in the same way that vehicles cannot be certified as safe to drive. Your mechanic may hand you a list of things that he or she has checked and perhaps even verified as safe. However, that does not mean that something will not go wrong once you get your car on the road or track.
Only mechanics that seek to rip you off will provide a fully automated safety inspection. It would be insane to think that in the next 50 years it would be ok to consider any application fully secure or safe, even with artificial intelligence. Even formal methods with formal specifications have been known to fail.
I suggest that you properly scope an application's unique situation before you determine the outcomes.
Sufficient for what?
Sufficient to defend against other purely automated attacks like worms? Probably.
Sufficient to defend against targeted attacks by skilled attackers? Certainly not.
A better question is "Does an automated scan mitigate risk to an acceptable level for my application?" The answer is, of course, it depends...
To use an analogy, you wouldn't ask the question, "Is going to my General Practitioner sufficient?" without the context of "I need a yearly checkup" or "I just chainsawed off half my torso."
An automated scan might be sufficient for a low criticality brochureware site on its own domain, but is definitely not sufficient for a banking app. Most sites fall somewhere in the middle, so... It depends ;-)