How to calculate our application security debt?

Question

Application security debt has some similarities to technical debt but there are few differences that we need to think about when deciding if our security debt load has gotten too high and needs to be paid off. I would like to know how to calculate security debts in our bank applications?

Answer 1

The best advice I have heard is to figure out a list of all of the different types of events that could occur as a result of your security debt. Next, try to estimate the cost of each of those events. Next, figure out the likelihood of those events occurring per year. Your final formula should look something like

Probability(event type 1) * Cost(event type 1) + 
Probability(event type 2) * Cost(event type 2) + 
... +
Probability(event type N) * Cost(event type N)

For example, let's say you determine that there are two issues which could be exploited by your security debit: SQL Injection + CSRF. (I've made up numbers to make the math easier):

• We expect 5 successful SQL injection attacks per year, each of which would have a recovery cost of $100,000
• We expect 10 successful CSRF attacks with a recovery cost of $25,000 apiece

Your estimated cost of security debt for the year in question would be:

(5 * $100,000) + (10 * $25,000) = $750,000

Answer 2

At this point, there is no standardised method of calculating the size (inventory) of technical debt. I have been working with a research team made up of PhD researchers from the University of Glasgow and MIT in order to start to create a framework to address this. We are combining MIT's Systems Theoretic Process Analysis for Security (STPA-Sec) and the concepts from Naval ship architecture, known as Vulnerability Design. While the techniques are intended to analyse an organisation and any sub-processes, it is also suited for a single application as a target for analysis.

The following is under development and testing. The concepts are useful nonetheless.

Systems-Theory Vulnerability Engineering

Calculating "Application Security Debt", to my team, is just another form of "Systems-Theory Vulnerability Engineering". This is different from the typical vulnerability management that can be addressed with automated scanners and patches and configuration. Instead, it looks at the "system" in question (your application, in this case) in its full context of people, processes, and technology as it connects to other systems. From this systems-theory perspective, you then determine where the vulnerabilities and weaknesses (near-future vulnerabilities) are.

These vulnerabilities might be across the spectrum of:

• SQLi hidden behind mitigations (bare and exposed SQLi vulnerabilities are an issue, not a debt)
• unpatched subsystems
• manual patching processes that require human intervention to trigger and complete
• lack of auditing

All of these things represent a "security debt" or a "systems vulnerability".

Note that this approach is not concerned with threats although vulnerabilities can be defined through an understanding of threats. This is not a Threat Modelling process (see the last section).

Step 1: Vulnerability Analysis (hyper-condensed form)

1. Define the security problem (what are you worried about?)
2. Identify types of unsecure control (e.g. program logic, system maintenance, assurance)
3. Identify causes of unsecure control types (e.g. processes, technology, resources, knowledge, culture, etc.)
4. Determine if those causes currently exist (constant state or intermittent)

You end up with a systemic analysis of your system's current vulnerabilities.

But now you still need to determine if you need to do something about it.

Step 2: Response Control Analysis (hyper-condensed form)

1. Determine the response/recovery controls around each identified vulnerability (can we detect and respond to insecure events?)
2. Determine which response/recovery controls cannot sufficiently contain any incidents that exploit or are caused by the vulnerabilities
3. Determine if the sufficient response/recovery controls suffer from current vulnerabilities that could result in insufficient control.

You end up with a list of security vulnerabilities with insufficient mitigations. This is your debt.

Note that not all of the items that result from this process are technological (in fact, from our initial case studies, few items are technological). You might find that your SQLi issue is actually a weakness in code review processes that are the result of a dev culture of feature-focus and not code quality focus. The debt, in this case, is cultural.

Step 3: Risk Alignment

This is where you start to design the trade-offs between 1) reducing vulnerabilities in various ways (people or processes or technology) and 2) improving response controls so that the goals of the system can be supported.

Just like any risk mitigation process, you need to keep mitigation costs lower than the expected losses and it all has to be completed to support the system's goals.

Vulnerability Modelling vs Threat Modelling

By taking a systems-theory and vulnerability-focused approach, we have found that this process promotes remedies that are cost-effective and targets the root cause of problems, not the effects of problems. It will also identify areas that need to be removed in order to reduce vulnerabilities (a subtractive process).

Threat-focused approaches tend to be reactive, expensive, technological, and additive (there's a new threat, we need more things!). This has the effect of creating more debt, not reducing it.

Answer 3

Estimating Application Security Debt During software development process, strict programming procedures, code analysis and application testing needs to be conducted. These procedures are particularly meant to ensure the developed application is of the highest quality possible and all the security flaws are sealed. However, despite the strict adherence to the set development procedures, once in a while the developed application becomes vulnerable. The accumulation of these vulnerabilities in an application eventually becomes the application security debt. In his article, Chris describes application debt as “*

Security debt is similar to technical debt. Both debts are design and implementation constructions that have negative aspects that aggregate over time and the code must be re-worked to get out of debt. Security debt is based on the latent vulnerabilities within an application. Application interest rates are the real world factors outside of the control of the software development team that lead to vulnerabilities having real cost. These factors include the cost of a security breach and attacker motivation to discover and exploit the latent vulnerabilities

*.” Application security debt financial model Currently, there are several application security debt metrics that tries to estimate the value of security debt. In this article, I will address a post by Russell that I believe helps to address security debt concept.

In his post, Russell indicates that *

“Application Security Debt is a ‘loan’ with variable principal which could range from 0% to 100% of your original project costs. The ‘principal’ is what you’ll eventually have to pay to fix security bugs or rewrite the code. It also has varying and uncertain ‘interest costs’, which are the costs of security breaches due to these vulnerabilities. This includes the possibility of the mother-of-all balloon payments (i.e. a huge loss event).”

* This gives us the debt formula
Debt = Expected Principal + Interest costs
The main task involves estimating the values of expected principal and the interest costs.

Expected Principal One thing we should note is that the value of expected principal will always be a percentage of the initial project cost. That means that the expected principal can a value between 0%-100% of the initial cost. This means that the expected principal value can increase with a discrete value F. The management are faced with several scenarios i.e, they can either: Do no re-write (0%), Do minor re-write (10%), Do major re-write (25%), Do substantial re-write (50%) or Pay a for a100% re-write. The expected principal will therefore be a proportion of the initial project cost F multiplied with the probability of the management choosing that option.

EP=∑_(i=1)^5▒〖p(F¬¬i).c(Fi)〗

Where F¬i=code fix scenario (i=0..n=5) P(Fi)=probability of choosing a particular scenario C(Fi)=cost of fixing the chosen scenario. However, the Expected Principal will vary depending on the industry, the company size and company resources among other factors. The management will determine the amount of resources to invest in different circumstances.

Interest costs The interest costs includes the cost involved in covering breach cost. They are incurred after an application’s vulnerability is exploited. Estimating the type of likelihood and the likelihood of an application breach occurring is hard. This is because there is no direct relationship that dictates which type of breach will occur on particular applications.

Currently, it is basic knowledge that intruders are targeting applications handling sensitive client data but at the same time, the security level in these applications handling sensitive data is very high. Due to the dynamic nature of security breaches, a vulnerability in one application might affect the security of another applications. For example, many organizations prefer to integrate third party applications with in-house applications. The integration of these applications might affect the security of another application. Therefore, it will be difficult to estimate the likelihood.

Use of data to estimate the interest costs might be useful in estimating the value of the interest cost. Data from different companies in the same industry can be used to determine how much a company will incur to cater for the data breach.