I've spent many hours on whitebox testing to make sure my code was safe. From a theoretical standpoint, the code SHOULD be safe. I then used several widely known testing tools (including one that cost $3500) to test for SQL injections, cross site scripting, CRLF injections, file handling security, and more. All software came up with 0 vulnerabilities found. Can I assume my app is 99% secure at this point? Has any of you ever done blackbox testing and thought it was safe only to find out otherwise?
4
-
let me put it this way, in a good penetration test report,30%-40% of the elements are retrived from automated tools. You may have 0 vulns. for now, or your tools may be unable to find them. And sometimes it goes the other way, tools find some vulnerabilities, which do not exists -aka false positive-.– cengizUzunCommented Jan 7, 2014 at 13:20
-
2Let's put it this way: You've proven with high certainity that your web app can not be hacked with the tools you tried. It is impossible to prove security.– PhilippCommented Jan 7, 2014 at 15:08
-
1The only secure computer is one that's unplugged, locked in a safe, and buried in a concrete block 100 feet below ground. And even then it's not perfectly secure. NOTHING is perfectly secure.– Mark E. HaaseCommented Jan 7, 2014 at 21:48
-
2There is no such thing as "99% secure". If you have a building with 100 doors, and 1 of them is unlocked, does that mean you are 99% secure? Agreed with @Philipp: you've proven that your web app can't be hacked with the tools you've used, but empirically it's inevitable that there is something you've missed. Most important is ensuring that there are no single points of security failure (known as defense in depth) — no single oversight should be enough to compromise whatever you're trying to protect.– Stephen TousetCommented Jan 7, 2014 at 23:17
Add a comment
|
5 Answers
Automated testing is a good step, but it isn't a replacement for a skilled human ethical hacker to find vulnerabilities. This may be through black and white box testing, code review, and specialized tools like metasploit and custom-written code. However it's done, people with the skills and experience dig far deeper and get many more results than automated tools no matter how good.
It's impossible to give a figure for how secure you are, if all you have done is used automated tools then you have protected yourself from a good deal of the automated exploit tools out there, but a skilled person will likely have a much better chance to find anything.
-
7
Whilst you've got a couple of answers indicating that indeed you can't be sure that your application is secure based on the output of automates tests, it might be useful to illustrate some problems that an automated tool would likely miss, that can still be quite serious.
Authorisation flaws are usually the canonical example of things that are hard to find with a straight scanner. For example assuming you have multiple levels of user privilege a situation where one type of user should not be able to access a specific piece of functionality which turns out to be available would be potentially quite a serious security concern but a straight scanner wouldn't necessarily recognise what it should or should not be able to access
Business logic flaws. A classic example is the ability on an e-commerce shop to buy something for -0.99 units of your local currency. I've seen this one in live tests but again the scanner lacks the context to be able to say that that's a problem.
Something as basic as a hidden directory can fool black-box scanners. They will look for them but for example if you had a hidden directory with an unprotected administration interface at /my_totally_secret_dir_honest_guv/ a scanner likely wont locate it.
Second order injection issues. For example your front-end application takes input from a user, but never represents it to the user. Instead it's presented in a different application (e.g. helpdesk applications where the ticket is sent to an administrator for work). If the 1st application doesn't echo the content, scanners can't find things like XSS which may still be present (although of course it could be argued that it's the other applications problem here)
That's some examples, there are more. What I'd suggest is that code review is valid for finding some of this and also manual review is very useful. A good resource to get more of an idea of the possibilities is the Web App Hackers Handbook
answered Jan 7, 2014 at 12:03
-
2One note: If the automated tools don't find the problem, then you're 99% safe from script kiddies, which is likely to be the majority of attacks. Commented Jan 7, 2014 at 15:09
-
indeed you're likely to be relatively safe from people using scanners if you use scanners (and fix the findings). on the percentage of attacks that are like that one thing to watch is that if your in a high risk industry (e.g. anything to do with bitcoins at the moment) then the percentage of motivated attackers could well be higher than 1% Commented Jan 7, 2014 at 17:56
I'm not sure about 99% secure, all I can say is it's not 100% secure. Any tool or pen tester can only test for currently known vulnerabilities and exploits. Security issues may come to light in future weeks/months/years that result in your app being compromised. For all you know there may be attackers out there now who know of an exploit not covered in your tests.
All you can say is that you've taken all reasonable steps to ensure the security of your application.
answered Jan 7, 2014 at 7:33
-
I disagree that
Any tool or pen tester can only test for currently known vulnerabilities and exploits- fuzzing doesn't necessarily need to target a known exploit. Certainly this kind of testing is a measure of the tools and the operator as much as the target.– symcbeanCommented Jan 7, 2014 at 9:48 -
Isn't fuzzing itself not just an exploit that you're testing against? Commented Jan 7, 2014 at 10:35
-
well, you can not stop fuzzing? -unless it is being done really fast and IPS blocks the fuzzer- You can always find zero day exploits or doing the samething in a different way... (xss is known, it is blocked by encoders and you bypass encoders maybe?) Commented Jan 7, 2014 at 13:24
-
I see 2 aspects to fuzzing though. The validation/sanitisation of input and then the performance aspect. You can deal with each of these separately. Much like you can't stop someone trying SQLi/XSS/CSRF, you can only stop it having a negative effect. Perhaps my thinking is a bit off but that's how I view it. Commented Jan 7, 2014 at 13:59
No, you can't say your application is 99% secure, 100% or 25% secure. You can only confirm that your application has passed a penetration test by and the penetration test reported 0 non-conformities. That sounds good, but you can't infere a percentage of security from here moreover with this information.
The technology in order to certify that a piece of code has not any security vulnerability has not been fully developed. Researches only have accomplished to certify that some kind of vulnerabilities does not exist in code with some characteristics like C code that does not allocate memory dinamically but in general, is very very difficult to certify that a piece of code has not any security vulnerability.
If you need such percentage, you can perform an OSSTMM compliant penetration test where the attack surface is calculated and then the RAVs. That is a way of calculating something similar to the percentage you mention and following a repeteable methodology.
In order to give proper value to this particular penetration test, it is important to know what exactly have been tested (the scope) and how (methodology, tools, etc.). If your provider have delivered to you the list of tools and outputs, that informatioh can be analyzed by another expert in case you need repeteability (interesting for any audit).
If the penetration test is based in automatic tools, as commented by other experts, my opinion is that you can be sure that you can rely in this penetration test. Automatic tools have important limitations and when they does not detect any vulnerability further analysis should be know to be sure why.
If you have proper documentation of what been tested and the results, if in the future you have a security incident, you can review the documentation to discover why the vulnerability was not detected and improve your pentesting methodology.
What you performed was vulnerability testing at best. Automated testing tools, like the ones that you describe simply test for known vulnerabilities. A penetration test has the human component to it, where the attacker would not only locate known vulnerabilities, but uncover potential new ones, and then exploiting them.
Although you took a great first step, you should take into consideration the investment that you have made into developing your application, and then determine if an actual penetration test is something worth exploring from a risk standpoint.
