Take the 2-minute tour ×
Information Security Stack Exchange is a question and answer site for Information security professionals. It's 100% free, no registration required.

Today I experienced a situation where a person responsible for the security of a company required a pentesting company to withdraw a clause in the contract that says that:

"during the pentest there exist the possibility to delete or modify sensitive data in the production environment unintentionally due to the execution of some tools, exploits, techniques, etc."

The client says that he is not going to accept that clause and that he believes that no company would accept that clause. He thinks that during a pentest information could be accessed but never deleted or modified.

We know that the execution of some tools like web crawlers or spiders can delete data if the web application is very badly programmed, so the possibility always exists if those types of tools are going to be used.

I know that these are the conditions of the client, and should be accepted, but:

Can a skilled and professional pentester always assure that no data will be deleted or modified in production during a pentest?

Can a pentest really be done if the pentest team has the limitation that data cannot be created nor modified?

Should the pentesting company always include the disclaimer clause just in case?

share|improve this question
52  
This reminds me of a wtf story where Google indexed a page that had a link that reset the entire database when it was clicked. Google just blindly went and indexed the page, followed the link, and wiped the database. There is no way to guarantee that a pentester might not do the same thing by accident. –  Mark Henderson Jul 17 '13 at 22:41
12  
Who knows what kind of logic bomb you might stumble upon. It's CYA against malicious traps and horrible misconfigurations that were a hidden disaster waiting to take out data the first time somebody triggered the domino string. Basically a "Sorry you don't have working backups, but that issue predates my breathing on your system" clause. –  Fiasco Labs Jul 18 '13 at 1:02
18  
@woliveirajr - thedailywtf.com/Articles/The_Spider_of_Doom.aspx and thedailywtf.com/Articles/WellIntentioned-Destruction.aspx - my memory wasn't quite spot on, but close enough –  Mark Henderson Jul 18 '13 at 3:18
42  
Surely a client insisting on this clause being removed is not being unreasonable, but rather not understanding why this clause is there. They think you will do something to break their system during testing. Just explain to them that if their code has some awful bug in it that deletes the database when you click "save", you obviously can't be held responsible for it. As often happens, this sounds like a human communication failure, not a technical question. –  MGOwen Jul 18 '13 at 4:16
7  
@tom-oconnor Even more so, better to lose data in a controlled and prepared way with recent backups than at a random time with some hacker slashing his way through. If data loss during pentesting is even an issue, it's propbably the least of the company's worries. And in this light I side with the remark that it's also more a communication problem than a technical problem. –  Grimace of Despair Jul 18 '13 at 9:30

7 Answers 7

up vote 125 down vote accepted

There's no way that a pentester can 100% assure that data will not be modified or deleted, in the same way as they can't assure that system availability won't be affected (I've knocked systems over with a port scan or a single ' character). as you say a web crawler can delete data from a system if it's been set-up badly.

I'd say that what should be said is something like "every care will be taken to ensure that the testing does not negatively affect the systems under review and no deliberate attempts will be made to modify or delete production data or to negatively affect the availability of in-scope systems. However with all security testing there is a risk that systems will be affected and the customer should ensure that backups of all data and systems are in place prior to the commencement of the review"

share|improve this answer
32  
I knocked a server over by putting a unicode character into a field in the customer's web app. You can never be 100% sure your actions won't affect uptime - unexpected failure vectors are everywhere. –  Polynomial Jul 17 '13 at 20:06
10  
I have had members of teams testing accounts provided by the client as 'test' accounts accidentally make international payments :-) It happens. Keep the clause in! –  Rory Alsop Jul 17 '13 at 22:04
21  
I don't know a lot about security but I've met a lot of messed up code bases and I approve this message. –  Erik Reppen Jul 18 '13 at 1:52
31  
As a penetration tester, you're essentially and ultimately going to be spending a lot of time poking at bits of the system with the question of "what happens when I input this unexpected data here?", and unless you're allowed to analyze the code itself at the same time there's no way to absolutely guarantee that the answer won't turn out to be "the system shits itself, erases its backend database and corrupts the backups before escaping across the border to Tijuana with the company budget". –  Shadur Jul 18 '13 at 7:53
2  
+1 for the improved phrasing. –  Ilmari Karonen Jul 18 '13 at 11:26

A pentester who claims that he will never alter production data is either a filthy liar, or thinks himself to be a lot more competent than he really is, or strongly intends to do nothing at all (that's the only surefire way of never breaking anything). In any case, you don't want to work with that guy.

A prospective customer who believes that skilled pentesters will never damage the tested systems, and who refuses to work with pentesters unless they promise exactly that, is a customer who 1. lives in the fairy unicorn dreamland, and 2. is almost guaranteed to ever do business only with filthy liars. He is in for some disillusion at some point, probably in a rather spectacular way.

It is a moral imperative and also basic self-protection for pentesters to include clauses about possible breakage in their contracts. The risk of collateral damage is real, even if the pentester is very good at what he does (because damage does not come from how good the pentester is but from how bad the tested system was designed and implemented). A pentester could be sued for not having warned the customer.

A customer who refuses a contract with such a clause has another name: "trouble". It is usually better to skip such customers altogether.

share|improve this answer
15  
+1 client smells funny, back away. This is 100% standard language included in all pentest engagements. –  bobince Jul 17 '13 at 20:03
2  
Sounds to me like he read it as "We can do what we like to your system, and claim 'accident' when we screw up", rather than "We don't want to be sued because of an unforeseeable situation". But yeah, AVOID. –  deworde Jul 18 '13 at 7:57
    
@deworde Any clarification to the condition such as "we will take all reasonable steps to avoid breaking the system" can later be sued over their steps not being sufficiently reasonable. Depending in the impact of the damage they do, it could easily exceed what would otherwise be their bill, and require a significant increase in their fee (i.e.: "Why does pen testing cost $1milion?" "Because that is what we expect you could sue us for, there is a 900k discount if you agree not to sue"). –  Matt Jul 18 '13 at 14:22
    
@Matt I agree. It's a failure to comprehend, not a failure to convey. If this is the standard language, the contract really can't say anything else without massive risk with zero upside. At worst, you'd end up sending the message that you're willing to negotiate your culpability. –  deworde Jul 18 '13 at 21:31

Caveat: I am not a professional pentester. And I work in backup software.

Can a skilled and professional pentester always assure that no data will be deleted or modified in production during a pentest?

I would say no, there are no absolute guarantees something will not accidentally be deleted when you are trying to break things. However, that said, a pentester should generally try to exploit systems in a way that does not involve deleting data. For example, whilst everyone's favourite SQL statement is:

select * from users where userid='dave'; DROP ALL TABLES;

A more responsible approach would simply be to list all data:

select * from users where userid='2' OR 1=1;

Generally, I imagine most pen testers have developed a series of exploits of the non-destructive variety like the latter.

Javascript injection in its various forms can use simple proof of concept code such as alert("exploited you"); rather than genuine exploit code.

Can a pentest be really done if the pentest team has the limitation that data cannot be created nor modified?

I would say no. How would you show a proof of concept of stored XSS without being able to store your XSS in the database? This is invariably going to create new data.

There are numerous examples where an exploit requires writing data, even if only temporarily.

Should the pentesting company always include the disclaimer clause just in case?

Again, I am stretching my personal knowledge here, but I am going to say yes.

This comes back to the whole point of hiring a pen testing company in the first place, however. The point of a pen test is to identify risks that might need evaluating and fix them before the bad guys find them.

I would also hope any good pentester would ask, and any good business would have thought of, disaster recovery on some scale. Surely you should have a backup of some form on your systems such that you can restore as needed. You need this not just in case the pentest destroys you data, but in case you are really really breached by the bad guys, in which case you might want a non-contaminated backup to roll back to.

share|improve this answer
2  
+1 for pointing out backups –  borrel Jul 18 '13 at 18:35
    
All I am thinking here is that even alert("exploited you"); might bring down stuff if there's some "useful" script that uses the alert system and acts on strings starting with e.g. 'e' –  medivh Aug 12 '13 at 9:03
    
@medivh probably, yes. You'd hope the company in question would furnish the pentesters with some documentation about their system so the guys know how to execute something that doesn't break it... –  user2213 Aug 12 '13 at 10:05

I run penetration tests quite regularly on my sites.

The first time I ran one I ground the site to a halt and flooded their mail server with spam.

I then tweeked a few forms, to get rid of the spam and it allowed be to identify and fix all the other known holes, without grinding the server to a halt.

I was frankly amazed at the deviousness of the exploiters, I had to plug holes I hadn't considered.

But the thing was, prior to running the tests, I thought my sites were secure. I had followed best practices, as far as they went, for website design, but there were still problems.

Now, with the benefit of hindsight, I appreciate that testing may affect my production site.

So I plan ahead.

I make sure that everything is backed up first.

If the site is really, really critical, I will create a complete clone, then test that clone first.

I have learned from experience that if you create the clone, create it on a different server. Otherwise, when the clone grinds to a halt, the server will still affect your production environment.

But I still carry out my tests. How do you think hackers gain access to sites? They presumably run tests like these themselves, in an unauthorised fashion. So until your site is protected it will always be vulnerable. It is much better to get the tests done first, with the precautions (backup etc) in place, than have to pick up the pieces when you least expect it.

So, yes it is acceptable, but it is also predictable, and should be managed accordingly.

share|improve this answer
2  
you can put the test clone in a throttled VM if hardware is tight. this way when it grinds to a halt it only grinds with 25% of CPU and prod still has 75 to play with –  ratchet freak Jul 18 '13 at 10:20
    
you should never (pen)test on a production server (virualized or not) the vm application can have bugs(ex: leak mem in kernel-space) –  borrel Jul 18 '13 at 18:39
    
If you create the clone on a separate server using the backup you took, you are also putting your Disaster Recovery skills to the test. –  Facebook Answers Jul 18 '13 at 21:44
    
+1 for testing the clone. –  Paul Wagland Jul 21 '13 at 22:14

Probably you can assure that only under certain conditions. Obviously Pentest is trade-off between many factors, some factor are directly antagonist to system availability and one of those is the depth of your analysis. Some people could argue that a pentest without exploitation isn't a pentest.

In the past I had to pentest some SCADA systems which are quite famous for their fragility and I can assure you it is possible to tune most of the tools to be gentle enough. For example disabling NMAP fingerprint and incresing delay between packets helped a lot.

However to answer your question, personally, I will NEVER agree to put a clause to guarantee that to a client of mine. I will give him my word but business-wise you can't be responsable if their application breaks by itself during your pentest.

share|improve this answer
    
+1 for the idea that a pentest without exploitation is not a pentest –  kinunt Jul 19 '13 at 10:20
    
management just sees Penetration Test as "expensive" and Vulnerability Assessment as "cheap". If they pay you enough is ALWAYS a pentest. –  Francesco Manzoni Jul 19 '13 at 10:33

"Can a skilled and professional pentester always assure that no data will be deleted or modified in production during a pentest?"

No.

"Can a pentest really be done if the pentest team has the limitation that data cannot be created nor modified?"

Don't think so..

"Should the pentesting company always include the disclaimer clause just in case?"

An internal agreement is going to look much different from a third party agreement. I don't know of any pen test company that doesn't have limitations of liability insurance, among other protections...

share|improve this answer

The answer to your title is "Yes". The answers to the 3 questions in the body of your post are: (1) no, you can't make any guarantees about others code (2) a limited form of pen testing might be done without INTENTIONALLY modifying or creating data (if you exclude any logging that the app may do) but the results would not be complete, and finally (3) you should ALWAYS include this disclaimer -- as much for the clients benefit as yours.

This disclaimer is not just a case of CYA, it is also vital information the client needs in order to prepare for the pentest. I am not a pen tester, but IMO this should't be buried on the last page in the fine print, but should be on page 1 of the contract, front and center and in a big bold font. The client company needs to prepare for things to go wrong -- backups need to be made, recovery plans created and put into place, etc.

The tester is basically looking for a flaw to exploit and can in no way guarantee that they won't find a flaw that does damage. Consider a typical software disclaimer for something that is NOT intended to encounter or create flaws, and then consider that you probably didn't write the code you are testing...

share|improve this answer
    
So, the answer should be "Yes", shouldn't it? Because it is acceptable (and expected) to modify production data during a pentest. –  kinunt Jul 22 '13 at 6:34
    
It would be unusual, but not irresponsible, to have a contract where you are not to intentionally modify production data in a non-standard way. It is technically impossible to guarantee that an unknown code base would not do damage upon NORMAL use, let alone the kind of unusual use that is involved in pen testing. I see that I should expand upon my answer a bit. –  jmoreno Jul 22 '13 at 16:49

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.