2

There are some formulas for risk-assessing security vulnerabilities, like the DREAD formula (Damage, Reproducibility, Exploitability, Affected users, Discoverability -- rate the vuln in each category on a scale from 1 to 10, then add them all up). Is there any evidence that these work better than just having a security professional look at the vulnerability as a whole and say, "Yeah, on a scale of 1 to 5, I give that a 3"?

Here's why I'm skeptical: The times that a formula is useful is when I'm doing something that is out of my area of expertise, like buying a used car, because I don't know anything about what cars are worth. So I would probably end up using a formula that involved the make and model of the car, the number of miles, and any visible damage.

But in assessing security vulnerabilities, the determination of severity is presumably going to be made by someone who has a good intuition of how bad a vulnerability is. In most cases that I've seen, when two experts independently judged how bad a vulnerability was, they came close to the same answer. And when they didn't agree, they certainly weren't going to resolve the disagreement by looking at the formula -- the person whose assessment was different from the formula, would just say that this is one of those times that the formula doesn't work. So I don't know what is the point of the formula. Formulas are useful when non-experts need to make a decision, but the people making decisions about security vulnerabilities shouldn't be non-experts!

So is there any evidence that the formulas have been useful, compared to the intuitive approach? Pretty open-ended about what counts as "evidence", but I'm thinking for example if a company tried the intuitive approach once, and tried the formula approach once, and found that one worked better than the other.

Improve this question
2
  • 1
    Not everyone on a team is going to have the same experience. Quantifying a vulnerability is more of a communication tool than anything. If there is mistrust between team members, that will hamper communication regardless of any quantifying framework. More issues in tech are people problems, than technical.
    – nbering
    Commented Jul 29, 2018 at 2:15
  • 1
    Also, even an expert is liable to be subject to various kinds of bias, e.g., a particularly novel vulnerability might seem more dangerous than an objective analysis would justify, while a boring one might seem less dangerous than it should. This sort of formulae, while not by any means eliminating bias, might be helpful in reducing it. (Not an answer, because I'm only speculating: if someone has evidence that this is/is not true, that would be much more useful.)
    – Harry Johnston
    Commented Jul 29, 2018 at 2:24

2 Answers 2

Reset to default
1

CVSS scores are meant as a guideline not as the gospel.

Proper risk assessment for your environment should always be determined by a review of what is considered critical infrastructure, what is more AT RISK of being exploited, etc.

If i have a device that cannot be updated due to required vendor software for business functionality, it has 10 critical vulnerabilities with a rating of 10 each.

But that device is only plugged in once a month for 30 minutes at non-regular intervals. I would say that device is a low to medium risk for my network.

Now if I have a device that is in the DMZ and has multiple vulnerabilities that are easily exploitable but aren't considered a "Critical" threat rating. I would evaluate that as being higher risk because it is exposed to the public

Edit:

Good additional resources to check out from Rapid7 (creators of metasploit): changing criticality of an asset, Adjusting risk with criticality, Risk scoring FAQ

Improve this answer
3
  • I think all these statements are true but not sure how it's an answer to my question. My question is: is there ever any advantage to using a formula to evaluating a security vulnerability (like the DREAD formula), compared to just having a security professional take everything into account and say, "I give this an 8 out of 10."
    – Bennett
    Commented Oct 26, 2018 at 18:46
  • The answer is you should use a formula and evaluate based on your environment
    – Matt
    Commented Oct 26, 2018 at 18:50
  • But that's the question: Why use a formula? Is there any evidence that this works better than just trying the security professional to assign a rating, without using a "formula"?
    – Bennett
    Commented Oct 28, 2018 at 3:25
0

I think you missed the point of the formula. The usefulness is not in the final score (that's for comparison to other vulnerabilities) but in the individual scores in each area.

As a professional, if you come to me with a vulnerability and tell me "yeah, it's a score of X", that's utterly useless to me. What I want to know are the component scores. Exploitable, discoverable, with high damage? Yeah, I want to get on that today. High damage that's difficult to discover and difficult to exploit? I'll patch in the next cycle.

DREAD and CVSS gives me those. How they come up with their final score is not interesting to me, but I want to know what the experts think in each area of the score. And I'm not going to assume that I am expert enough or that I have someone working for me that is expert enough to make that call. Tell me, high, medium, low, what the vuln is in each area.

Improve this answer
5
  • That sounds right, but still, same question: Is there evidence that the formula works better than just having a security expert eyeball it and make a judgment call? Of course each individual component of DREAD is important. But I would assume that when an expert looks at a vulnerability and makes an instant judgment call, all of those factors are baked in intuitively.
    – Bennett
    Commented Nov 25, 2018 at 20:39
  • What evidence would suffice? It's inherently subjective and qualitative.
    – schroeder ♦
    Commented Nov 25, 2018 at 20:40
  • Evidence would be an experiment where: two groups of researchers consider the same vulnerability and assessing the priority of fixing it. One group applies the formula. The control group just eyeballs it. At the end, ask: In which group does there seem to be more in-group agreement? And do the groups agree with each other, on average? If the groups agree on average, that suggests the formula doesn't matter. If the formula group has less in-group agreement, then the formula isn't even helping converge on any answer, much less the correct one.
    – Bennett
    Commented Nov 25, 2018 at 20:46
  • That process will not result in a repeatable experiment. My answer is about the uselessness of an overall score. You appear to only be concerned about the aggregate score where intuition is baked in. I say that no matter how you arrive at that score, it is meaningless.
    – schroeder ♦
    Commented Nov 25, 2018 at 20:51
  • I don't know how this is not a "repeatable experiment". You can always repeat the experiment with the same vulnerability, and new groups of people, the same way you'd repeat any other experiment involving people. (You could also repeat the experiment with a different vulnerability -- perhaps some vulnerabilities, by their nature, result in more agreement.) (I should have been more clear: when I say "group", I still mean that the individuals all make their assessments individually, but the grouping is for comparison. Not for making a "group decision".)
    – Bennett
    Commented Nov 26, 2018 at 21:12

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .