3

How effective are tools such as w3af in looking for web app vulnerabilities compared to looking for vulnerabilities manually? Are they able to find all vulnerabilities from OWASP top 10 such as reflected xss, persistent xss, sqli, lfi/rfi, and unrestricted file upload? Or will some vulnerabilities fall through the cracks and remain undiscovered?

2

Tools are always limited in my opinion. Testing an authorization mechanism and session fixation (I've yet to come a cross a tool that can do this) are just two examples a tool can't (properly) do. Tools are good for the so called low hanging fruits (most of the OWASP top 10 issues), but can't be called a penetration test.

I generally use tools to get a global indication of the system. While running in the background I perform manual tests. In my experience as a security consultant, the number of vulnerabilities discovered by web application scanners is always less than what I find manually.

Another thing is combining vulnerabilities, for example:

  1. Session cookie does not have the httpOnly attribute set
  2. Application is vulnerable to CSRF attacks
  3. Application is vulnerable to Persistent XSS attacks

While no 1 by itself can be considered low and no 2 and 3 are high, combining the three will lead to session hijacking, which can be considered critical. This is something a tool will not do.

So, yes most vulnerabilities will discover OWASP top 10 flaws. And yes, it is possible that some vulnerabilities will not be discovered, this can be due to time restrains.

Also keep in mind that what's considered sufficiently secured today can be non-secure next month.

| improve this answer | |
2

Manual testing:

  • Slow;
  • Humans can forget some important tests;
  • Complex, sophisticated and creative testing.

Automated testing:

  • Fast;
  • Tests a lot (thousands) of vulnerabilities (never forget a test);
  • Easily repeatable after remediation;
  • "Cake Recipe" (will just test what it was designed to test).

Based on the above points, we can say that each type of testing have its own positives and negatives sides.

Are they able to find all vulnerabilities from OWASP top 10 such as reflected xss, persistent xss, sqli, lfi/rfi, and unrestricted file upload?

A lot of Web Security Scanners were designed to find all the above types of vulnerabilities, but nothing can guarantee that they will find all instances of the above vulnerabilities.

Or will some vulnerabilities fall through the cracks and remain undiscovered?

Some vulnerabilities can fall through the cracks, no tool is perfect.

| improve this answer | |
  • Add to your list that the automated tests are more easily repeatable after remediation. – schroeder Nov 29 '14 at 0:40
  • @schroeder Done. – Lucas NN Dec 3 '14 at 5:47
1

How effective are tools such as w3af in looking for web app vulnerabilities compared to looking for vulnerabilities manually?

They are effective for what they are. Scanners will find the low hanging fruit.

However, they typically fail at finding vulnerabilities caused by flawed business logic. In this area humans have the advantage.

Scanners typically test 10's of thousand of vulnerabilities which is not feasible for a single person to accomplish. In this area scanners have an advantage.

Are they able to find all vulnerabilities from OWASP top 10 such as reflected xss, persistent xss, sqli, lfi/rfi, and unrestricted file upload?

Yes.

Webinspect has an OWASP scan if you want to restrict scanning to the OWASP top 10.

Or will some vulnerabilities fall through the cracks and remain undiscovered?

Yes.

Scanners will not always find the same amount or type of vulnerabilities every time. At least this has been my experience using Webinspect for over 10 years. Meaning that scan result "A" may contain 10 instances of XSS where scan results "B" may contain 7 instances of XSS.

Usually when I see this happen it's the same page in case A and B, but B lists fewer affected parameters.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.