Dark Reading is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Risk

6/25/2013
11:47 PM
50%
50%

Vulnerability Severity Scores Make For Poor Patching Priority, Researchers Find

A bug's Common Vulnerability Scoring System (CVSS) score doesn't necessarily correlate with whether the vulnerability is being used in attacks

Relying on the measure of vulnerability severity to prioritize what to patch and what to put off for another day is a waste of effort on software flaws that pose no danger, while missing others that are being exploited, according to two researchers that plan to reveal their findings at the Black Hat Security Briefings later this year.

Click here for more of Dark Reading's Black Hat articles.

The research analyzed the severity of vulnerabilities as ranked on the popular Common Vulnerability Scoring System (CVSS) with the existence of exploits and whether those exploits were being used in the wild to attack systems. The researchers -- Luca Allodi, a Ph.D. student in the field of security economics at the University of Trento in Italy, and Fabio Massacci, professor of information systems and security at UT -- found that the CVSS score did not correlate strongly with the attribute that arguably matters most to companies: whether the vulnerability is being used to attack systems.

"The CVSS could be high, but you may have a low risk of being exploited, while you can get a low CVSS score and still be attacked," Massacci says. "There is not much correlation between the CVSS only and the chance of being attacked."

The Common Vulnerability Scoring System uses a number of qualitative characteristics of a software flaw to determine the severity of the vulnerability on a 10-point scale. The CVSS combines a number of metrics -- such as the complexity of the attack and whether it impacts a system's confidentiality, integrity, and availability -- to come up with the score.

The researchers compared CVSS scores from the National Vulnerability Database (NVD) with information from the Exploit Database on the subset of vulnerabilities for which exploits had been created and with information from Symantec on the vulnerabilities that were actually being targeted by attackers in the wild.

Vulnerabilities targeted by exploits for sale in the underground should be patched immediately, as there was a strong correlation between the sale of an exploit for a particular vulnerability and the danger of that vulnerability being attacked. However, there was less correlation between the existence of a proof-of-concept attack in the Exploit Database and the risk of attack.

The complexity of the attack -- one of the metrics used to make up the CVSS score -- also appears to have a stronger correlation to the chance of a vulnerability being targeted by attackers than the overall score itself, the researchers say.

"If your vulnerability is in an exploit kit, then patch," Allodi says. "And if it is easy to exploit, then patch. But if it is difficult -- more complex -- to exploit, then it depends on the importance of the software with a vulnerability."

[With flaw tallies varying by up to 75 percent, vulnerability data needs to be taken with a grain of salt, yet reports based on the data fail to include caveats, Black Hat presenters say. See Don't Take Vulnerability Counts At Face Value.]

Many of the criticisms echo those of researcher Dan Guido, co-founder and CEO of security startup Trail of Bits, who argued that companies should focus on which vulnerabilities are being attacked and find simple defenses that defeat the attacks. In a 2011 study of vulnerabilities targeted by popular exploit kits, for example, Guido found two mitigations that could block 90 percent of the attacks.

Doing that sort of analysis with CVSS scores is impossible, he says. The scores do not provide enough information to the information security managers, especially because two aspects of an attack are only known by the potential victim.

"The vendor has no idea what the company's network looks like and what the attacker might be after," Guido says. "And without those two critical pieces of information, it's hard to make the CVSS score relevant."

While the research highlights that CVSS has weaknesses, the scoring system is a good standard by which companies can express a single severity for software flaws, says Wolfgang Kandek, chief technology officer of vulnerability management firm Qualys. While Qualys does not use CVSS as the measure of severity for software flaws in its own service, the framework is good for the majority of companies, he says.

"It depends on the level of sophistication," Kandek says. "Our customers are good with our severity, and I know that some very sophisticated customers can pull apart CVSS values to make their own decision, but for most companies the straight score is a good measure."

Yet for companies who are trying to find the best use of their resources, focusing on CVSS scores to prioritize patching will waste effort, argues UT's Massacci. In many ways, prioritizing patching based on CVSS scores is like triaging patients in an emergency room by just their temperature, he says.

"A single number is not a good idea -- CVSS is like measuring you for a temperature and then sending you to the operating room if it's high," he says. "What you should do, like in the medical domain, is first measure if you have a fever, and then you do a blood test, and then you do an X-ray."

Have a comment on this story? Please click "Add Your Comment" below. If you'd like to contact Dark Reading's editors directly, send us a message. Veteran technology journalist of more than 20 years. Former research engineer. Written for more than two dozen publications, including CNET News.com, Dark Reading, MIT's Technology Review, Popular Science, and Wired News. Five awards for journalism, including Best Deadline ... View Full Bio

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
CraigSchiller
50%
50%
CraigSchiller,
User Rank: Apprentice
6/28/2013 | 2:06:18 PM
re: Vulnerability Severity Scores Make For Poor Patching Priority, Researchers Find
This research is a good example of fallacious reasoning. Of course exploit databases derived from exploit kits show stronger correlation to attacks in the wild, especially when compared to a vulnerability database that is trying to document all vulnerabilities. Similarly, if you had chosen as your criteria the degree of coverage of known vulnerabilities the exploit databases would perform poorly. It's a reasonable expectation that a list of exploits in exploit kits would perform better than a list of known exploits because the exploits in exploit kits have a distribution system and individual exploits may or may not. If you would have concluded that it would be useful to include a good exploitability score along with things like CVSS scores and asset criticality ratings then I might have agreed. It's like Black or White photography <smile>, "and" is much better.</smile>
amanion
50%
50%
amanion,
User Rank: Apprentice
6/26/2013 | 10:48:46 PM
re: Vulnerability Severity Scores Make For Poor Patching Priority, Researchers Find
CVSS does support temporal and environmental vectors and scores, however
few sites publishing CVSS scores go beyond the base (as intended, the
user is supposed to provide temporal and environmental scores). It would be interesting to perform the same analysis on complete (base+temporal+environmental) CVSS scores. Also, I don't expect the Exploitability subscore to predict attacks, I expect it to measure the relative ease of attack. Widespread attacks usually depend on large target populations. In CVSS Target Distribution is an environmental metric.
anon7395245893
50%
50%
anon7395245893,
User Rank: Apprentice
6/26/2013 | 8:45:29 PM
re: Vulnerability Severity Scores Make For Poor Patching Priority, Researchers Find
Scoring systems will continue to evolve and should continue to evolve just as we have seen metrics other domains like fitness & diet change over the years. The good news is that CVSS clearly states which facets are being measured and represented - use it if it is useful, don't use it if it is not. The research above is accurate and insightful but going in anyone can see that CVSS is missing the sufficient modeling of the threat environment and without it, the cost to the adversary cannot be faithfully represented. In my opinion, over the next 3 to 5 years, many models will need to be developed and if done right, they will be modular and interoperable
anon7395245893
50%
50%
anon7395245893,
User Rank: Apprentice
6/26/2013 | 8:31:09 PM
re: Vulnerability Severity Scores Make For Poor Patching Priority, Researchers Find
Correction: CVSS is a 100 point scale from 0.0 to 10.0
cmdrfrog
50%
50%
cmdrfrog,
User Rank: Apprentice
6/26/2013 | 6:45:57 PM
re: Vulnerability Severity Scores Make For Poor Patching Priority, Researchers Find
A good opening point on the value of professional Risk Assessment. CVSS is only a severity scoring and does not equal risk because it has not been considered in the context of threats. Compliance driven organizations tend to just go with CVSS and not invest in a full risk or threat assessment becuase they are under regulatory or statutory requirement to "get compliant" regardless, so under that constraint its the best prioritization scheme in a bad situation.
Mobile Banking Malware Up 50% in First Half of 2019
Kelly Sheridan, Staff Editor, Dark Reading,  1/17/2020
7 Tips for Infosec Pros Considering A Lateral Career Move
Kelly Sheridan, Staff Editor, Dark Reading,  1/21/2020
For Mismanaged SOCs, The Price Is Not Right
Kelly Sheridan, Staff Editor, Dark Reading,  1/22/2020
Register for Dark Reading Newsletters
White Papers
Video
Cartoon Contest
Write a Caption, Win a Starbucks Card! Click Here
Latest Comment:   It's a PEN test of our cloud security.
Current Issue
IT 2020: A Look Ahead
Are you ready for the critical changes that will occur in 2020? We've compiled editor insights from the best of our network (Dark Reading, Data Center Knowledge, InformationWeek, ITPro Today and Network Computing) to deliver to you a look at the trends, technologies, and threats that are emerging in the coming year. Download it today!
Flash Poll
How Enterprises are Attacking the Cybersecurity Problem
How Enterprises are Attacking the Cybersecurity Problem
Organizations have invested in a sweeping array of security technologies to address challenges associated with the growing number of cybersecurity attacks. However, the complexity involved in managing these technologies is emerging as a major problem. Read this report to find out what your peers biggest security challenges are and the technologies they are using to address them.
Twitter Feed
Dark Reading - Bug Report
Bug Report
Enterprise Vulnerabilities
From DHS/US-CERT's National Vulnerability Database
CVE-2020-7245
PUBLISHED: 2020-01-23
Incorrect username validation in the registration processes of CTFd through 2.2.2 allows a remote attacker to take over an arbitrary account after initiating a password reset. This is related to register() and reset_password() in auth.py. To exploit the vulnerability, one must register with a userna...
CVE-2019-14885
PUBLISHED: 2020-01-23
A flaw was found in the JBoss EAP Vault system in all versions before 7.2.6.GA. Confidential information of the system property's security attribute value is revealed in the JBoss EAP log file when executing a JBoss CLI 'reload' command. This flaw can lead to the exposure of confidential information...
CVE-2019-17570
PUBLISHED: 2020-01-23
An untrusted deserialization was found in the org.apache.xmlrpc.parser.XmlRpcResponseParser:addResult method of Apache XML-RPC (aka ws-xmlrpc) library. A malicious XML-RPC server could target a XML-RPC client causing it to execute arbitrary code. Apache XML-RPC is no longer maintained and this issue...
CVE-2020-6007
PUBLISHED: 2020-01-23
Philips Hue Bridge model 2.X prior to and including version 1935144020 contains a Heap-based Buffer Overflow when handling a long ZCL string during the commissioning phase, resulting in a remote code execution.
CVE-2012-4606
PUBLISHED: 2020-01-23
Citrix XenServer 4.1, 6.0, 5.6 SP2, 5.6 Feature Pack 1, 5.6 Common Criteria, 5.6, 5.5, 5.0, and 5.0 Update 3 contains a Local Privilege Escalation Vulnerability which could allow local users with access to a guest operating system to gain elevated privileges.