Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Risk

6/25/2013
11:47 PM
50%
50%

Vulnerability Severity Scores Make For Poor Patching Priority, Researchers Find

A bug's Common Vulnerability Scoring System (CVSS) score doesn't necessarily correlate with whether the vulnerability is being used in attacks

Relying on the measure of vulnerability severity to prioritize what to patch and what to put off for another day is a waste of effort on software flaws that pose no danger, while missing others that are being exploited, according to two researchers that plan to reveal their findings at the Black Hat Security Briefings later this year.

Click here for more of Dark Reading's Black Hat articles.

The research analyzed the severity of vulnerabilities as ranked on the popular Common Vulnerability Scoring System (CVSS) with the existence of exploits and whether those exploits were being used in the wild to attack systems. The researchers -- Luca Allodi, a Ph.D. student in the field of security economics at the University of Trento in Italy, and Fabio Massacci, professor of information systems and security at UT -- found that the CVSS score did not correlate strongly with the attribute that arguably matters most to companies: whether the vulnerability is being used to attack systems.

"The CVSS could be high, but you may have a low risk of being exploited, while you can get a low CVSS score and still be attacked," Massacci says. "There is not much correlation between the CVSS only and the chance of being attacked."

The Common Vulnerability Scoring System uses a number of qualitative characteristics of a software flaw to determine the severity of the vulnerability on a 10-point scale. The CVSS combines a number of metrics -- such as the complexity of the attack and whether it impacts a system's confidentiality, integrity, and availability -- to come up with the score.

The researchers compared CVSS scores from the National Vulnerability Database (NVD) with information from the Exploit Database on the subset of vulnerabilities for which exploits had been created and with information from Symantec on the vulnerabilities that were actually being targeted by attackers in the wild.

Vulnerabilities targeted by exploits for sale in the underground should be patched immediately, as there was a strong correlation between the sale of an exploit for a particular vulnerability and the danger of that vulnerability being attacked. However, there was less correlation between the existence of a proof-of-concept attack in the Exploit Database and the risk of attack.

The complexity of the attack -- one of the metrics used to make up the CVSS score -- also appears to have a stronger correlation to the chance of a vulnerability being targeted by attackers than the overall score itself, the researchers say.

"If your vulnerability is in an exploit kit, then patch," Allodi says. "And if it is easy to exploit, then patch. But if it is difficult -- more complex -- to exploit, then it depends on the importance of the software with a vulnerability."

[With flaw tallies varying by up to 75 percent, vulnerability data needs to be taken with a grain of salt, yet reports based on the data fail to include caveats, Black Hat presenters say. See Don't Take Vulnerability Counts At Face Value.]

Many of the criticisms echo those of researcher Dan Guido, co-founder and CEO of security startup Trail of Bits, who argued that companies should focus on which vulnerabilities are being attacked and find simple defenses that defeat the attacks. In a 2011 study of vulnerabilities targeted by popular exploit kits, for example, Guido found two mitigations that could block 90 percent of the attacks.

Doing that sort of analysis with CVSS scores is impossible, he says. The scores do not provide enough information to the information security managers, especially because two aspects of an attack are only known by the potential victim.

"The vendor has no idea what the company's network looks like and what the attacker might be after," Guido says. "And without those two critical pieces of information, it's hard to make the CVSS score relevant."

While the research highlights that CVSS has weaknesses, the scoring system is a good standard by which companies can express a single severity for software flaws, says Wolfgang Kandek, chief technology officer of vulnerability management firm Qualys. While Qualys does not use CVSS as the measure of severity for software flaws in its own service, the framework is good for the majority of companies, he says.

"It depends on the level of sophistication," Kandek says. "Our customers are good with our severity, and I know that some very sophisticated customers can pull apart CVSS values to make their own decision, but for most companies the straight score is a good measure."

Yet for companies who are trying to find the best use of their resources, focusing on CVSS scores to prioritize patching will waste effort, argues UT's Massacci. In many ways, prioritizing patching based on CVSS scores is like triaging patients in an emergency room by just their temperature, he says.

"A single number is not a good idea -- CVSS is like measuring you for a temperature and then sending you to the operating room if it's high," he says. "What you should do, like in the medical domain, is first measure if you have a fever, and then you do a blood test, and then you do an X-ray."

Have a comment on this story? Please click "Add Your Comment" below. If you'd like to contact Dark Reading's editors directly, send us a message. Veteran technology journalist of more than 20 years. Former research engineer. Written for more than two dozen publications, including CNET News.com, Dark Reading, MIT's Technology Review, Popular Science, and Wired News. Five awards for journalism, including Best Deadline ... View Full Bio

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
CraigSchiller
50%
50%
CraigSchiller,
User Rank: Apprentice
6/28/2013 | 2:06:18 PM
re: Vulnerability Severity Scores Make For Poor Patching Priority, Researchers Find
This research is a good example of fallacious reasoning. Of course exploit databases derived from exploit kits show stronger correlation to attacks in the wild, especially when compared to a vulnerability database that is trying to document all vulnerabilities. Similarly, if you had chosen as your criteria the degree of coverage of known vulnerabilities the exploit databases would perform poorly. It's a reasonable expectation that a list of exploits in exploit kits would perform better than a list of known exploits because the exploits in exploit kits have a distribution system and individual exploits may or may not. If you would have concluded that it would be useful to include a good exploitability score along with things like CVSS scores and asset criticality ratings then I might have agreed. It's like Black or White photography <smile>, "and" is much better.</smile>
amanion
50%
50%
amanion,
User Rank: Apprentice
6/26/2013 | 10:48:46 PM
re: Vulnerability Severity Scores Make For Poor Patching Priority, Researchers Find
CVSS does support temporal and environmental vectors and scores, however
few sites publishing CVSS scores go beyond the base (as intended, the
user is supposed to provide temporal and environmental scores). It would be interesting to perform the same analysis on complete (base+temporal+environmental) CVSS scores. Also, I don't expect the Exploitability subscore to predict attacks, I expect it to measure the relative ease of attack. Widespread attacks usually depend on large target populations. In CVSS Target Distribution is an environmental metric.
anon7395245893
50%
50%
anon7395245893,
User Rank: Apprentice
6/26/2013 | 8:45:29 PM
re: Vulnerability Severity Scores Make For Poor Patching Priority, Researchers Find
Scoring systems will continue to evolve and should continue to evolve just as we have seen metrics other domains like fitness & diet change over the years. The good news is that CVSS clearly states which facets are being measured and represented - use it if it is useful, don't use it if it is not. The research above is accurate and insightful but going in anyone can see that CVSS is missing the sufficient modeling of the threat environment and without it, the cost to the adversary cannot be faithfully represented. In my opinion, over the next 3 to 5 years, many models will need to be developed and if done right, they will be modular and interoperable
anon7395245893
50%
50%
anon7395245893,
User Rank: Apprentice
6/26/2013 | 8:31:09 PM
re: Vulnerability Severity Scores Make For Poor Patching Priority, Researchers Find
Correction: CVSS is a 100 point scale from 0.0 to 10.0
cmdrfrog
50%
50%
cmdrfrog,
User Rank: Apprentice
6/26/2013 | 6:45:57 PM
re: Vulnerability Severity Scores Make For Poor Patching Priority, Researchers Find
A good opening point on the value of professional Risk Assessment. CVSS is only a severity scoring and does not equal risk because it has not been considered in the context of threats. Compliance driven organizations tend to just go with CVSS and not invest in a full risk or threat assessment becuase they are under regulatory or statutory requirement to "get compliant" regardless, so under that constraint its the best prioritization scheme in a bad situation.
Stop Defending Everything
Kevin Kurzawa, Senior Information Security Auditor,  2/12/2020
Small Business Security: 5 Tips on How and Where to Start
Mike Puglia, Chief Strategy Officer at Kaseya,  2/13/2020
5 Common Errors That Allow Attackers to Go Undetected
Matt Middleton-Leal, General Manager and Chief Security Strategist, Netwrix,  2/12/2020
Register for Dark Reading Newsletters
White Papers
Video
Cartoon
Current Issue
6 Emerging Cyber Threats That Enterprises Face in 2020
This Tech Digest gives an in-depth look at six emerging cyber threats that enterprises could face in 2020. Download your copy today!
Flash Poll
How Enterprises Are Developing and Maintaining Secure Applications
How Enterprises Are Developing and Maintaining Secure Applications
The concept of application security is well known, but application security testing and remediation processes remain unbalanced. Most organizations are confident in their approach to AppSec, although others seem to have no approach at all. Read this report to find out more.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2019-20477
PUBLISHED: 2020-02-19
PyYAML 5.1 through 5.1.2 has insufficient restrictions on the load and load_all functions because of a class deserialization issue, e.g., Popen is a class in the subprocess module. NOTE: this issue exists because of an incomplete fix for CVE-2017-18342.
CVE-2019-20478
PUBLISHED: 2020-02-19
In ruamel.yaml through 0.16.7, the load method allows remote code execution if the application calls this method with an untrusted argument. In other words, this issue affects developers who are unaware of the need to use methods such as safe_load in these use cases.
CVE-2011-2054
PUBLISHED: 2020-02-19
A vulnerability in the Cisco ASA that could allow a remote attacker to successfully authenticate using the Cisco AnyConnect VPN client if the Secondary Authentication type is LDAP and the password is left blank, providing the primary credentials are correct. The vulnerabilities is due to improper in...
CVE-2015-0749
PUBLISHED: 2020-02-19
A vulnerability in Cisco Unified Communications Manager could allow an unauthenticated, remote attacker to conduct a cross-site scripting (XSS) attack on the affected software. The vulnerabilities is due to improper input validation of certain parameters passed to the affected software. An attacker ...
CVE-2015-9543
PUBLISHED: 2020-02-19
An issue was discovered in OpenStack Nova before 18.2.4, 19.x before 19.1.0, and 20.x before 20.1.0. It can leak consoleauth tokens into log files. An attacker with read access to the service's logs may obtain tokens used for console access. All Nova setups using novncproxy are affected. This is rel...