Is it OK to do a security testing in parallel with a QA testing?

Question

This may not be a very technical question per se, so please feel free to move it to the respective forum/stack. However, I would really appreciate anybody experienced or experiencing a similar situation and the solutions being adopted.

So, I am working as a security engineer with a small start-up. Now, in my previous organization, a typical corporate, I had my boss to take care of doing the convincing etc. with the top level management and getting the things required for us to work smoothly. However, in my current role, being a start-up, I am the only engineer responsible for the information security of the whole company. Now as overwhelming, as it gets, it is becoming an amazing learning curve for me by the day. I am picking up different aspects of security as baby steps, one at a time, starting off with the most important assets that need to be protected and that lie at the maximum risk/in the threat zone. I am trying to prioritize stuff and see where a pen test is needed and where is a VA needed.

There are many challenges that I am facing of which I would like to the discuss one below: Now given the above situation that I am in, one major challenge that I am facing as of now is that I constantly keep pushing by tech leads to make sure that every every new feature that I am supposed to test needs to first get QA verified before I can pick it up for a VA/PT/security testing. The reason being, there have been a couple of instances, where due to an incomplete QA on the feature, my security test cases gave false negatives, as the feature itself was broken contrary to the expectation. Now this is something that the tech leads are not finding convincing enough to allow security testing only after QA approval. They need more examples/test cases to support this Their argument being,

security test can be done in parallel to QA and that I need not really wait on the QA as it is simply pushing the release date un-necessarily.

Am I right in my argument ? If yes, what else can I possibly do to convince them on my points. If am not right, please help me understand why is my argument incorrect.

Answer 1

I currently work as a developer, with some responsibility for sprint planning etc. so my answer is based mostly from a 'how should testing be accounted for' approach. I'm going to talk about agile here, but the same principal applies.

I can't see anything wrong in principal with testing in parallel to QA, assuming that it's understood that anything which fails QA will need re-testing.

Assuming the business assigns estimates to work (think agile development), dev time, QA time and security testing time should be added together to give the total estimated time for a task to be completed. Without going into too much depth on agile development, the sprint is made up of a number of tasks (cards), which the total estimate for fits into the sprint length.

Once a card is complete, it goes to QA and for security testing. If either of those two steps fail, it will go back in for rework etc.

Now, in an ideal world, all the cards will pass QA and security testing. What actually happens is some cards pass and some will fail (either at QA or security testing). Assuming the business doesn't turn around and just accept the risk, there will be some rework and the card should go back to QA and security.

By running your testing in parallel, you should save some time in the sprint. When cards fail QA, the business needs to understand/accept that the card will require re-testing from QA and security. You may end up spending the full estimated time re-testing, or less/more time. That time then affects the sprint and the release date.

In order to argue either side, you should look at how much rework you usually get. If most of your cards fail QA, then it make sense that you do your security testing once QA has signed off. If most of your cards pass QA, then it makes sense to run your testing in parallel.

I've based this answer off agile, where everything should have an estimate etc. The same principle applies regardless, each stage of development will have a time cost associated with it, even if its not written down anywhere.

My suggestion would be to look at how many cards usually fail QA and work out if it's beneficial to run testing in parallel or not.

Answer 2

"due to an incomplete QA on the feature, my security test cases gave false negatives"

I don't really understand what you are trying to say here. Testing does not change the functionality, security, capacity or performance of the system. However the whole point of testing is to identify where the system does not deliver on any of these aspects (along with other attributes such as usability) which should then prompt changes to the code / architecture / configuration. The changes should go back through the testing cycle.

The only really justifiable arguments for the order in which each aspect is evaluated are costs and time. Earlier testing may save costs later in the process if they fail - but will have to be repeated when the code is reworked. Hence the testing which is likely to fail should be done early balanced against favouring the testing which is cheaper to do earlier in the cycle against the availability of resources to do the testing.

Whoever is last in the process before a feature goes live is always going be unpopular. Deliberately casting yourself in this role might be inferred as an inflated sense of self-importance.

Answer 3

It's not okay

If you software guards the nuclear launch codes, penetration testing MUST be performed on the exact release binary that will go into production. That is for the obvious reason that bug fixes can introduce vulnerabilities. A last-minute commit is the perfect place for a Russian spy to hide the code for a back door.

Except when it is

If your software does not guard the nuclear launch codes, there is room for compromise, in the interest of saving time. The penetration test team could agree to start its test cycle prior to code freeze, for example, as long as the development team avoids making any high risk commits afterward. This would be need to be verified by the QA lead by inspecting the source code history prior to certifying the penetration test results. The development team will need to explain each of the commits that took place after penetration testing started. A bug fix involving a style change might be okay to ignore. A change to authentication configuration might not. If anything is deemed too high a risk, a second round of penetration testing may be required, or the product owner will need to sign off on the risk of going to production without having fully penetration tested the software. Some businesses are okay with that, and that is their prerogative.

Ideally, though, your penetration tests would be fully automated, making this whole topic a non-issue.

Answer 4

You can do security testing once you're done with your QA testing because security testing is most beneficial once you're done with the smooth functioning of the user interface.