How to know when a pen test is complete?

Question

Specifically considering client websites where we have been asked to execute a pen test; at what point do we stop and say we're done?

We have access to various tools (some automated, some manual); but if we say "we tried all our tools, and couldn't make any progress", that could be construed as us saying that we're not clever enough (and there's always some hacker out there who could be cleverer).

So; how do we protect ourselves against upset clients who claim that we didn't work with due diligence? Is there a standard report framework we can work within?

Answer 1

So this is actually a very interesting question for the industry in general. The way I would suggest you handle it is

• Have something in your contract that disclaims liability for vulnerabilities not noted during testing. Reason for this is, it's basically impossible to be sure that you've found every exploitable issue in a website, or any other system. To pick one example, think of all the sites that were sitting vulnerable to shellshock for years and years, should all the pen test companies who touched one of those sites be liable for not telling their customers?

• Have a methodology, saying what you will do. This should cover the general areas of testing that will be completed. For websites, consider basing on something like the OWASP Top 10 as a starting point. This gives you some common ground with the customer on what you'll be looking at.

• Make sure your company covers the basics with a checklist. as @rapli says above document all the little things, but don't overblow the severity. Whilst it's important to make sure your test isn't just a checklist, using one can avoid embarassing mistakes where basic tests get missed.

The problem you might/will run into is unrealistic expectations from customers. that one is a case by case to address. If you get a customer that expects that their complex application will be completely reviewed in like 5 person-days of testing, well you should explain why that's not a practical concept :)

Answer 2

Specify in contract which security aspects you investigate and only take responsibility for those. You wont always find vulnerabilities. But I guarantee you will find few minor things, and I suggest you to include every little detail you can in the report, missing HSTS in headers, weak ciphers, etc. So they see that you did something.

There are some reporting tools I know of, but they are either not publicly available or paid products.

Answer 3

As addition to the other answers:

When your contract states your scope and which vulnerabilities to test, you could write down that you actually tested cases in scope against known vulnerabilities defined in the contract. Something like:

SQL injection:

SQL-Injections are ...

We identified 42 Input fields in the Webpages in Scope. We identified 0 
Input fields vulnerable to SQL injection (with our used method).

Drawback is: in case there actually is a SQL-Injection possible in one of the listed fields appended to the report; you didn't promise 100% security but promised that this isn't a vulnerability, but then you surely made something wrong - that's why you should somehow state which kind of tests you used, what kind of SQL-Injections are state of the art in the introduction to SQL-Injections, so you will not be blamed for not finding attacks which will be found in future.

This also works for general Website-Security stuff:

HTTPS: https is ... recommended to use TLS X.Y ...

We determined the usage of https for all websites in Scope with TLS X.Y.
Certificates expire Dates are set to Date X which is in the recommended
certificate expire time range. Certificates hold an 4096-Bit key which is
acceptable for current usage.

Also try to create templates of your reports since you don't want to rewrite everything about SQL-Injection and HTTPS/TLS etc. over and over again, so you only have to fill the results of your tests. This will also ensure you made all tests and did not miss any, when you see some paragraph not written to be done and not found anything nor have any findings.