Is it acceptable that a skilled professional pentester deletes or modifies sensitive data in production unintentionally during a pentest?

Question

Today I experienced a situation where a person responsible for the security of a company required a pentesting company to withdraw a clause in the contract that says that:

"during the pentest there exist the possibility to delete or modify sensitive data in the production environment unintentionally due to the execution of some tools, exploits, techniques, etc."

The client says that he is not going to accept that clause and that he believes that no company would accept that clause. He thinks that during a pentest information could be accessed but never deleted or modified.

We know that the execution of some tools like web crawlers or spiders can delete data if the web application is very badly programmed, so the possibility always exists if those types of tools are going to be used.

I know that these are the conditions of the client, and should be accepted, but:

Can a skilled and professional pentester always assure that no data will be deleted or modified in production during a pentest?

Can a pentest really be done if the pentest team has the limitation that data cannot be created nor modified?

Should the pentesting company always include the disclaimer clause just in case?

Answer 1

There's no way that a pentester can 100% assure that data will not be modified or deleted, in the same way as they can't assure that system availability won't be affected (I've knocked systems over with a port scan or a single ' character). as you say a web crawler can delete data from a system if it's been set-up badly.

I'd say that what should be said is something like "every care will be taken to ensure that the testing does not negatively affect the systems under review and no deliberate attempts will be made to modify or delete production data or to negatively affect the availability of in-scope systems. However with all security testing there is a risk that systems will be affected and the customer should ensure that backups of all data and systems are in place prior to the commencement of the review"

Answer 2

A pentester who claims that he will never alter production data is either a filthy liar, or thinks himself to be a lot more competent than he really is, or strongly intends to do nothing at all (that's the only surefire way of never breaking anything). In any case, you don't want to work with that guy.

A prospective customer who believes that skilled pentesters will never damage the tested systems, and who refuses to work with pentesters unless they promise exactly that, is a customer who 1. lives in the fairy unicorn dreamland, and 2. is almost guaranteed to ever do business only with filthy liars. He is in for some disillusion at some point, probably in a rather spectacular way.

It is a moral imperative and also basic self-protection for pentesters to include clauses about possible breakage in their contracts. The risk of collateral damage is real, even if the pentester is very good at what he does (because damage does not come from how good the pentester is but from how bad the tested system was designed and implemented). A pentester could be sued for not having warned the customer.

A customer who refuses a contract with such a clause has another name: "trouble". It is usually better to skip such customers altogether.

Answer 3

Caveat: I am not a professional pentester. And I work in backup software.

Can a skilled and professional pentester always assure that no data will be deleted or modified in production during a pentest?

I would say no, there are no absolute guarantees something will not accidentally be deleted when you are trying to break things. However, that said, a pentester should generally try to exploit systems in a way that does not involve deleting data. For example, whilst everyone's favourite SQL statement is:

select * from users where userid='dave'; DROP ALL TABLES;

A more responsible approach would simply be to list all data:

select * from users where userid='2' OR 1=1;

Generally, I imagine most pen testers have developed a series of exploits of the non-destructive variety like the latter.

Javascript injection in its various forms can use simple proof of concept code such as alert("exploited you"); rather than genuine exploit code.

Can a pentest be really done if the pentest team has the limitation that data cannot be created nor modified?

I would say no. How would you show a proof of concept of stored XSS without being able to store your XSS in the database? This is invariably going to create new data.

There are numerous examples where an exploit requires writing data, even if only temporarily.

Should the pentesting company always include the disclaimer clause just in case?

Again, I am stretching my personal knowledge here, but I am going to say yes.

This comes back to the whole point of hiring a pen testing company in the first place, however. The point of a pen test is to identify risks that might need evaluating and fix them before the bad guys find them.

I would also hope any good pentester would ask, and any good business would have thought of, disaster recovery on some scale. Surely you should have a backup of some form on your systems such that you can restore as needed. You need this not just in case the pentest destroys you data, but in case you are really really breached by the bad guys, in which case you might want a non-contaminated backup to roll back to.

Answer 4

I run penetration tests quite regularly on my sites.

The first time I ran one I ground the site to a halt and flooded their mail server with spam.

I then tweeked a few forms, to get rid of the spam and it allowed be to identify and fix all the other known holes, without grinding the server to a halt.

I was frankly amazed at the deviousness of the exploiters, I had to plug holes I hadn't considered.

But the thing was, prior to running the tests, I thought my sites were secure. I had followed best practices, as far as they went, for website design, but there were still problems.

Now, with the benefit of hindsight, I appreciate that testing may affect my production site.

So I plan ahead.

I make sure that everything is backed up first.

If the site is really, really critical, I will create a complete clone, then test that clone first.

I have learned from experience that if you create the clone, create it on a different server. Otherwise, when the clone grinds to a halt, the server will still affect your production environment.

But I still carry out my tests. How do you think hackers gain access to sites? They presumably run tests like these themselves, in an unauthorised fashion. So until your site is protected it will always be vulnerable. It is much better to get the tests done first, with the precautions (backup etc) in place, than have to pick up the pieces when you least expect it.

So, yes it is acceptable, but it is also predictable, and should be managed accordingly.

Answer 5

The answer to your title is "Yes", and the client needs to know this!

This disclaimer is not a case of CYA, it is instead vital information the client needs in order to prepare for the pentest. I am not a pen tester, but IMO this should't be buried on the last page in the fine print, but should be on page 1 of the contract, front and center and in a big bold font. The client company needs to prepare for things to go wrong -- backups need to be made, recovery plans created and put into place, etc. Letting the client believe that a pen test is risk free is irresponsible. You are by definition attempting to trigger incorrect behavior, with no control over how far that broken behavior extends or does.

The answers to the 3 questions in the body of your post are: (1) no, you can't make any guarantees about others code (2) a limited form of pen testing might be done without INTENTIONALLY modifying or creating data (if you exclude any logging that the app may do) but the results would not be complete, and finally (3) you should ALWAYS include this disclaimer -- as much for the clients benefit as yours.

The tester is basically looking for a flaw to exploit and can in no way guarantee that they won't find a flaw that does damage. Consider a typical software disclaimer for something that is NOT intended to encounter or create flaws, and then consider that you probably didn't write the code you are testing...

Although the story may be fictional, consider The Spider of Doom!, not a pen tester and with no expectations of causing damage. Bad implementation and, boom, massive data loss.

Answer 6

Probably you can assure that only under certain conditions. Obviously Pentest is trade-off between many factors, some factor are directly antagonist to system availability and one of those is the depth of your analysis. Some people could argue that a pentest without exploitation isn't a pentest.

In the past I had to pentest some SCADA systems which are quite famous for their fragility and I can assure you it is possible to tune most of the tools to be gentle enough. For example disabling NMAP fingerprint and incresing delay between packets helped a lot.

However to answer your question, personally, I will NEVER agree to put a clause to guarantee that to a client of mine. I will give him my word but business-wise you can't be responsable if their application breaks by itself during your pentest.

Answer 7

"Can a skilled and professional pentester always assure that no data will be deleted or modified in production during a pentest?"

No.

"Can a pentest really be done if the pentest team has the limitation that data cannot be created nor modified?"

Don't think so..

"Should the pentesting company always include the disclaimer clause just in case?"

An internal agreement is going to look much different from a third party agreement. I don't know of any pen test company that doesn't have limitations of liability insurance, among other protections...

Answer 8

Penetration testing should follow a methodology where tests that can cause damage are done only in non-production scenarios, such as a staging or lab environment.

The real issue is not deletion or modification of data because testers can leave those test cases to non-production systems. The real issue is leaving behind new data, such as error logs or superfluous erroneous files that reveals confidential information, or even the existence of the pentest itself, including the penetration tester's client-side data material.

Another issue prevalent in penetration testing is the revelation of other clientele data, work email, or browser bookmarks during demos or screencasts.