3

This may not be a very technical question per se, so please feel free to move it to the respective forum/stack. However, I would really appreciate anybody experienced or experiencing a similar situation and the solutions being adopted.

So, I am working as a security engineer with a small start-up. Now, in my previous organization, a typical corporate, I had my boss to take care of doing the convincing etc. with the top level management and getting the things required for us to work smoothly. However, in my current role, being a start-up, I am the only engineer responsible for the information security of the whole company. Now as overwhelming, as it gets, it is becoming an amazing learning curve for me by the day. I am picking up different aspects of security as baby steps, one at a time, starting off with the most important assets that need to be protected and that lie at the maximum risk/in the threat zone. I am trying to prioritize stuff and see where a pen test is needed and where is a VA needed.

There are many challenges that I am facing of which I would like to the discuss one below: Now given the above situation that I am in, one major challenge that I am facing as of now is that I constantly keep pushing by tech leads to make sure that every every new feature that I am supposed to test needs to first get QA verified before I can pick it up for a VA/PT/security testing. The reason being, there have been a couple of instances, where due to an incomplete QA on the feature, my security test cases gave false negatives, as the feature itself was broken contrary to the expectation. Now this is something that the tech leads are not finding convincing enough to allow security testing only after QA approval. They need more examples/test cases to support this Their argument being,

security test can be done in parallel to QA and that I need not really wait on the QA as it is simply pushing the release date un-necessarily.

Am I right in my argument ? If yes, what else can I possibly do to convince them on my points. If am not right, please help me understand why is my argument incorrect.

2

I currently work as a developer, with some responsibility for sprint planning etc. so my answer is based mostly from a 'how should testing be accounted for' approach. I'm going to talk about agile here, but the same principal applies.

I can't see anything wrong in principal with testing in parallel to QA, assuming that it's understood that anything which fails QA will need re-testing.

Assuming the business assigns estimates to work (think agile development), dev time, QA time and security testing time should be added together to give the total estimated time for a task to be completed. Without going into too much depth on agile development, the sprint is made up of a number of tasks (cards), which the total estimate for fits into the sprint length.

Once a card is complete, it goes to QA and for security testing. If either of those two steps fail, it will go back in for rework etc.

Now, in an ideal world, all the cards will pass QA and security testing. What actually happens is some cards pass and some will fail (either at QA or security testing). Assuming the business doesn't turn around and just accept the risk, there will be some rework and the card should go back to QA and security.

By running your testing in parallel, you should save some time in the sprint. When cards fail QA, the business needs to understand/accept that the card will require re-testing from QA and security. You may end up spending the full estimated time re-testing, or less/more time. That time then affects the sprint and the release date.

In order to argue either side, you should look at how much rework you usually get. If most of your cards fail QA, then it make sense that you do your security testing once QA has signed off. If most of your cards pass QA, then it makes sense to run your testing in parallel.

I've based this answer off agile, where everything should have an estimate etc. The same principle applies regardless, each stage of development will have a time cost associated with it, even if its not written down anywhere.

My suggestion would be to look at how many cards usually fail QA and work out if it's beneficial to run testing in parallel or not.

1

"due to an incomplete QA on the feature, my security test cases gave false negatives"

I don't really understand what you are trying to say here. Testing does not change the functionality, security, capacity or performance of the system. However the whole point of testing is to identify where the system does not deliver on any of these aspects (along with other attributes such as usability) which should then prompt changes to the code / architecture / configuration. The changes should go back through the testing cycle.

The only really justifiable arguments for the order in which each aspect is evaluated are costs and time. Earlier testing may save costs later in the process if they fail - but will have to be repeated when the code is reworked. Hence the testing which is likely to fail should be done early balanced against favouring the testing which is cheaper to do earlier in the cycle against the availability of resources to do the testing.

Whoever is last in the process before a feature goes live is always going be unpopular. Deliberately casting yourself in this role might be inferred as an inflated sense of self-importance.

  • I understand that early testing would help save time/cost. However, what I meant with the quoted statement above, was say for example, a certain feature was QA verified for a happy flow. However, due to release requirements, the non-happy flows were not tested for the feature. Now, when a security test was run on a non-happy flow of the feature, it did not trigger the existence of any security bug. With experience I was sure that there would be a certain bug in the feature. So some research revealed that the feature itself was broken for a non-happy flow and hence security tests failed. – qre0ct May 19 '16 at 7:05
  • From your description the security was NOT broken, the functionality was broken. If you can only test the security of code which is working as exected then you're not doing your job very well. – symcbean May 19 '16 at 8:22
  • ok, let me try another shot at it. Because the functionality was broken, I was getting false negatives for my security test cases. In other words, had the functionality been working as expected (for both happy and unhappy flows) my security test cases might have had more deterministic and apt results. – qre0ct May 19 '16 at 10:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.